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(57) ABSTRACT 

Frames of a digital video signal are spatially filtered to 
remove impulse and other noise from the video signal. The 
spatially filtered frame is temporally filtered to further 
remove noise from the digital video signal. The spatial filter 
is adaptive, heavily filtering portions of low detail in the 
represented subject matter while lightly filtering or not 
filtering portions of moderate to high detail in the repre- 
sented subject matter. As a result, clarity and detail in the 
subject matter of the digital video signal are preserved. The 
temporal filter is similarly adaptive and determines if and to 
what degree the subject matter of portions of the spatially 
filtered frame correlate with corresponding portions of a 
previous frame. By previously spatially filtering the frame, 
determinations regarding the correlation of the current frame 
with corresponding portions of the previous, which is also 
spatially and temporally filtered, are more accurate. 
Accordingly, temporal filtering can be performed more 
aggressively without temporally blurring the digital video 
signal. The result of spatial and temporal filtering of the 
digital video signal is significant reduction in noise without 
degradation of the signal and improved compression of the 
digital video signal 

13 Claims, 10 Drawing Sheets 
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SPATIAL AND TEMPORAL FILTERING of a particularly strong filter achieves greater compression at 

MECHANISM FOR DIGITAL MOTION the expense of greater signal loss, and a particularly light 

VIDEO SIGNALS filter preserves more of the original signal at the expense of 

a smaller degree of compression. Thus, digital video signals 

FIELD OF THE INVENTION 5 which include significant noise force a choice between 

The present invention relates to digital video signal fil- clarity and compression rate, i.e., the rate between the 

tering and, in particular, to a particularly efficient signal amouot of data re( i uircd 10 "present the digital video signal 

filtering mechanism for pre-processing digital video signals m uncompressed and compressed forms, 

prior to compression and encoding of the digital video In addition, noise introduced by inexpensive, low-cost 

signals for subsequent decoding and display. 30 video capture and processing equipment is not adequately 

addressed by currently available digital video signal filters. 

BACKGROUND OF THE INVENTION Currently available digital video signal filters typically focus 

With the advent of digital video products and services, on Gaussian noise. However, ever growing popularity of 

such as Digital Satellite Service (DSS) and storage and inexpensive video recording and processing equipment, 

retrieval of video streams on the Internet and, in particular, 15 such & video cameras, video tape, and personal computer 

the World Wide Web, digital video signals are becoming video <*Pto™ cards > of moderate or questionable quality 

ever present and drawing more attention in the marketplace. introduces noise which is not Gaussian. 

Because of limitations in digital signal storage capacity and What is needed is a digital video signal filter which can 

in network and broadcast bandwidth limitations, compres- better eliminate the types of noise found in digital video 

sion of digital video signals has become paramount to digital 20 signals without compromising the clarity and quality of the 

video storage and transmission. As a result, many standards image of the digital video signal to thereby simultaneously 

for compression and encoding of digital video signals have improve the quality and compression rate of the digital video 

been promulgated. For example, the International Telecom- signal, 
munication Union (ITU) has promulgated the H.261 and 

H.263 standards for digital video encoding. Additionally, the 25 SUMMARY OF THE INVENTION 

International Standards Organization (ISO) has promulgated i n accordance with the present invention, a digital video 

the Motion Picture Experts Group (MPEG), MPEG-1, and signal is spatially and temporally filtered to remove 

MPEG-2 standards for digital video encoding. unwanted noise of the digital video signal such that better 

These standards specify with particularity the form of 3Q image quality of the digital video signal and reduced size 

encoded digital video signals and how such signals are to be when compressed and encoded are simultaneously achieved, 

decoded for presentation to a viewer. However, significant The temporal filter determines whether a current block of 

discretion is left as to how the digital video signals are to be pixels represents the same subject matter represented by a 

transformed from a native, uncompressed format to the block of pixels of a previous frame and temporally filters the 

specified encoded format. As a result, many different digital 35 current block if the blocks represent the same subject matter, 

video signal encoders currently exist and many approaches By spatially filtering the current block prior to making such 

are used to encode digital video signals with varying degrees a determination, impulse noise is significantly reduced and 

of compression achieved. the accuracy with which such a determination is made is 

The primary objective of any digital video signal encoder enhanced substantially. Accordingly, the temporal filter can 

is to achieve a high degree of compression without a ^ more aggressively filter the digital video signal without 

significant loss of video signal. Video signal compression is risking temporal blurring of the digital video signal, 

generally achieved by representing identical or similar por- The spatial filter is adaptive in that a frame of the digital 

tions of an image as infrequently as possible to avoid video signal is divided into blocks and each block is filtered 

redundancy. As a result, an image which has only very according to the amount of subject matter detail represented 

coarse detail and very few distinct colors can be compressed 45 in the block. Blocks are chosen small enough to allow 

to a much smaller representation in comparison to a com- smoothing up to but not including edges in the represented 

pressed representation of an image with significant amounts subject matter and large enough to provide effective smooth- 

of very fine detail and many distinct colors. Unfortunately, ing. The amount of detail of the subject matter represented 

video cameras and other video signal acquisition equipment in each block is determined by measuring the variance of 

introduce noise into the video signal and, from the perspec- 50 pixels values of the block. A large variance indicates sig- 

tive of video signal processing, the noise is generally indis- nificant detail in the subject matter of the block and causes 

tinguishable from fine detail in the subject of the video the block to forego spatial filtering. A moderate variance 

signal. For example, ordinary noise in a monochromatic indicates moderate detail and/or significant noise in the 

image may be indistinguishable from the fine detail and subject matter of the block and causes the block to be lightly 

texture of a terrycloth towel photographed up close. 55 spatially filtered. A low variance indicates little or no detail 

Digital video signal compression typically involves a in the subject matter of the block and causes heavy spatial 

transformation, e.g., a discrete cosine transformation (DCT), filtering of the block. Since small blocks are evaluated 

in which pixels which are relatively close in value to one separately, background sections of an image can be filtered 

another are represented in a particularly compact form. heavily to remove nearly all noise up to and very near edges 

Noise in a digital video signal has a particularly adverse 60 at which detail in the subject matter of the image is not so 

effect on such compression since the noise is frequently heavily filtered and therefore preserved, 

unrelated to the subject matter of the video image and For blocks which are spatially filtered, a median filter 

frequently renders portions of the digital video signal inap- effectively removes impulse noise from the digital video 

propriate for representation in such a compact form. signal, and a smoothing filter further reduces any remaining 

To achieve both enhanced image quality and greater 65 noise. The median filter effectively removes the impulse 

compression, video signal encoders frequently filter a video noise without blurring or smudging the impulse noise 

signal prior to encoding the video signal. However, the use thereby removing the impulse noise without any affects 



03/04/2004, EAST version: 1.4.1 



US 6,281, 

3 

which are annoying or perceptible to a human viewer of a 
display of a decoded digital video signal. In addition, such 
effective removal of impulse noise substantially improves 
the performance of subsequent filtering steps such as 
smoothing and temporal filtering. After application of the 5 
median filter, lightly spatially filtered blocks are filtered with 
a light smoothing filter and heavily spatially filtered blocks 
are filtered with a heavy smoothing filter. 

After spatial filtering, the spatially filtered frame is adap- 
tively temporally filtered to further remove noise from the jq 
digital video signal. However, temporal filtering of the 
spatially filtered frame is bypassed altogether if the current 
frame and the previous frame are so different that common 
subject matter between the frames is unlikely to be found. 
When the spatially filtered frame is temporally filtered, the 15 
spatially filtered frame is divided into blocks. The blocks are 
sufficiently small to have a relatively good likelihood of 
matching corresponding blocks of a previous frame notwith- 
standing some changes and/or motion between the spatially 
filtered frame and the previous frame and are sufficiendy 2 o 
large to relatively accurately determine whether correspond- 
ing blocks do, in fact, represent the same subject matter. 
Each block is compared to a corresponding block of the 
previous frame to determine if the subject matter represented 
by the block of the spatially filtered frame is the same as the 25 
subject matter represented by the corresponding block of the 
previous frame. The spatial filtering described above sig- 
nificantly improves the accuracy with which such a deter- 
mination is made. 

If the corresponding blocks do not represent the same 30 
subject matter, the block of the spatially filtered frame is not 
temporally filtered. Conversely, if the corresponding blocks 
do represent the same subject matter, each pixel of the 
spatially filtered block is compared a corresponding pixel of 
the block of the previous frame to determine how closely the 35 
corresponding pixels represent the same subject matter. A 
large difference indicates low correlation between the cor- 
responding pixels, and thus low confidence that the corre- 
sponding pixels represent the same subject matter, and 
causes the pixel of the spatially filtered block to forego 40 
temporal filtering. A moderate difference indicates moderate 
correlation between the pixels and/or significant noise 
between the corresponding pixels and causes the pixel of the 
spatially filtered block to be lightly temporally filtered. A 
low difference indicates high correlation between the cor- 45 
responding pixels, and therefore high confidence that the 
corresponding pixels represent the same subject matter, and 
causes heavy temporal filtering of the pixel of the spatially 
filtered block. Since each pixel is evaluated separately and 
is filtered temporally, the image is not blurred spatially by 50 
the temporal filter. In addition, the multi-tiered approach to 
determining the similarity of the subject matter of the 
current frame and the previous frame with the accuracy of 
such a determination enhanced by the spatial filtering of the 
current frame and of the previous frame, having been 55 
previously spatially and temporally filtered, further pre- 
serves the temporal clarity and detail of the motion video 
image. 

The result of filtering frames of a motion video image in 
accordance with the present invention is that the particular 60 
types of noise prevalent in motion video images, e.g., 
impulse noise, is effectively removed from the motion video 
image without the spatial and temporal blurring and general 
sacrifice of image detail of conventional filtering mecha- 
nisms. Digital video signals are usually improved consider- 65 
ably such that artifacts introduced by recording and process- 
ing the digital video signal are removed and clarity appears 
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restored. Hie enhanced clarity is particularly noticeable in 
digital video signals representing stationary text. In addition, 
artifacts which are sometimes introduced into digital video 
signals during compression and encoding are mitigated since 
the digital video signal is largely free from unwanted noise. 
The filtering mechanism according to the present invention 
is also particularly robust in that virtually no type of digital 
video signal is worsened by application of the filtering 
mechanism according to the present invention. Conversely, 
most conventional filtering mechanisms improve some types 
of digital video signals and worsen other types of digital 
video signals. In addition, the lack of unwanted noise 
substantially reduces the amount of data required to repre- 
sent the digital video signal in a compressed form without 
sacrificing the image quality of the digital video signal. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of a video signal encoder which 
includes a digital video signal pre-filter in accordance with 
the present invention. 

FIG. 2 is a block diagram showing the digital video signal 
pre-filter of FIG. 1 in greater detail, including a spatial filter 
and a temporal filter. 

FIG. 3 is a block diagram of the spatial filter of FIG. 2 in 
greater detail. 

FIG. 4 is a logic flow diagram of the processing of the 
spatial filter of FIG. 3. 

FIG. 5 is a logic flow diagram of a heavy filtering step of 
the logic flow diagram of FIG. 4 in greater detail. 

FIG. 6 is a logic flow diagram of a light filtering step of 
the logic flow diagram of FIG. 4 in greater detail. 

FIGS. 7 A and 7B are block diagrams illustrating the 
relative positions of pixels used by the spatial filter of FIG. 
3. 

FIG. 8 is a block diagram of the temporal filter of FIG. 2 
in greater detail. 

FIG. 9 is a block diagram of the processing of the 
temporal filter of FIG. 8. 

FIG. 10 is a block diagram of a step of the logic flow 
diagram of FIG. 9 in greater detail. 

FIG. 11 is a block diagram of a computer system in which 
the video signal encoder of FIG. 1 operates. 

DETAILED DESCRIPTION 

In accordance with the present invention, a video signal 
encoder 100 (FIG. 1) pre- filters a digital video signal prior 
to encoding the digital video signal to thereby remove 
unwanted noise and simultaneously improve the image 
quality and reduce the size of the digital video signal as 
encoded. Video signal encoder 100 includes a pre-filter 130 
which, as described more completely below, includes a 
spatial filter 202 (FIG. 2) and a temporal filter 204. Both 
spatial filter 202 and temporal filter 204 adaptivcly filter out 
noise, of various types typically introduced into digital video 
signals, according to the subject matter of the digital video 
signal. The result is improved image quality of the digital 
video signal as encoded, often better than the image quality 
of the motion video image prior to encoding, and signifi- 
cantly improved compression rates at the same time. In other 
words, the distortion-rate performance of the encoding of the 
digital video signal is significantly enhanced. 

Appreciation of the present invention is enhanced by 
understanding of the various components and operation of 
video signal encoder 100 (FIG. 1). Video signal encoder 100 
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receives a frame of a video signal from a video source (not The quantization performed by quantizer 112 is controlled 

shown in FIG. 1) which can include, for example, a video by quantization parameter Q 114 which is in turn controlled 

camera, a video cassette player, a video laser disk player, or by Q adjuster 116 in a manner described more completely in 

similar video source. Video signal encoder 100 stores the u.S. patent application Ser. No. 08/819,507 for "Digital 

frame in buffer 102 after moving any frame previously 5 video Signal Encoder and Encoding Method" by Albert S. 

stored in buffer 102 into buffer 104. Thus, video signal Wang that description is incorporated herein by refer- 

encoder 100 stores two consecutive frames in buffers 102 ence 

and 104. The frame stored in buffer 102 is sometimes o *. jj j j.ir.ur 

referred to herein as the current frame, and the frame stored *»»»! ««)ded .ndependenUy of other frames 

in buffer 104 is sometimes referred to herein as the previous , n of dl 8 ltal Vlde ° »gnal while other frames are encoded are 

fj im& 10 motion-compensated and are therefore encoded in a form 

The current frame is filtered in pre-filter 130 in the manner *f hic ^ » dependent upon other frames of the digital video 

described more completely below to produce a current signal. In encoding frames of the digital video signal all 

filtered frame. I/P framer 106 of video signal encoder 100 frames are compressed by reducing redundancy of image 

includes a motion estimator/compensator 108 which data within a single frame, and motion-compensated frames 

retrieves the current filtered frame from pre-filter 130 and a 15 are further compressed by reducing redundancy of image 

reconstructed previous frame from a buffer 128 and derives data within a sequence of frames. Since a digital video signal 

motion vectors which represent motion between the current includes a sequence of images which differ from one another 

filtered frame and reconstructed previous frame. The recon- only incrementally, significant compression can be realized 

structed previous frame is reconstructed from a previously by encoding a number of frames as motion-compensated 

encoded frame as described more completely below. For 20 frames. However, noise in sequential images of a video 

each of one or more macrob locks of the current frame, signal can be misinterpreted as additional differences 

motion estimator 106 derives a motion vector which speci- between the images and can therefore reduce the efficiency 

fies a portion of the reconstructed previous frame which the ^th which motion-compensated frames can represent the 

macroblock corresponds and an associated motion vector sequential images, 

error signal. 25 Pre-Filter 130 

Motion estimator/compensator 108 produces a current . , . 

motion-compensated frame from the motion vectors and the , Since noise in a digital video signal is generally misin- 

current filtered frame received from pre-filter 130 and the ter P r f * 35 de \^ f m the ^ u ma " er .°* ^ <hgrtal video 

reconstructed previous frame received from buffer 128. Sl * n * ' rem ^ al of noise from . the digital video signal by 

Motion estimator/compensator 108 passes the motion- 30 P 1 ^ 161 ^° JF™ * enCOch ° g ±e dl& f V ? deo ^ 

compensated frame to transform coder 110 which performs proves he distortion-rate performance of such encoding, 

direct cosine transformation (DCT) on the motion- Pre-filter 130 is shown in greater detail in FIG. 2. 

compensated macroblocks of the motion-compensated Pre-filter 130 includes a spatial filter 202 and a temporal 

frame to produce a transformed frame. Transform coder 110 filter 204 - s P atial filter 202 filters P ixels of the cuncnl frame » 

passes the transformed frame to a quantizer 112. Quantizer 35 received from buffer 102, using only other pixels of the 

112 quantizes coefficients used in transform coder 110 and current frame. Temporal filter 204 receives the current frame 

these coefficients are then used later for Huf&nan coding the from s P atial filter 202 after spatial filtering. Temporal 

transformed frame to complete compression of the current filter 204 fe adaptive and first determines in a manner 

frame retrieved from buffer 102. Huffman coding is described more completely below whether corresponding 

described more completely in copending U.S. patent appli- 40 P ortions of the filtered current frame and the reconstructed 

cation Ser. No. 08/818,805 for "Method and Apparatus for previous frame, received from buffer 128, represent the 

Implementing Motion Detection and Estimation in Video same subject matter prior to temporally filtering the spatially 

Compression" filed on Mar. 14, 1997, and that description is filtered ™ mni frame * Prior s P alial filterin g of the frame 

incorporated herein by reference. substantially improves the accuracy with which temporal 

As described briefly above, a reconstructed previous 45 fi^r 204 makes such a determination, 

frame is used to estimate motion between consecutive Spatial Filter 202 

frames. The reconstructed previous frame is also used to Spatial filter 202 is shown in greater detail in FIG. 3. The 

filter the current frame in pre-filter 130 in a manner processing of spatial filter 202 is shown in logic flow 

described more completely below. The reconstructed previ- diagram 400 (FIG. 4). Spatial filter 202 (FIG. 3) is adaptive 

ous frame is formed as follows. A dequantizer 120 receives 50 in that spatial filter 202 filters respective portions of the 

the encoded current frame from quantizer 112 and performs current frame differently according to the subject matter 

the inverse of the quantization performed by quantizer 112. content of each portion. Accordingly, portions of the current 

The dequantized frame is transferred from dequantizer 120 frame which include very little detail, e.g., monochromatic 

to a transform decoder 122 which performs an inverse direct backgrounds, are filtered heavily, (ii) portions of the current 

cosine transformation of the DCT performed by transform 55 frame with moderate detail are filtered lightly, and (iii) 

coder 110. A frame reconstructor 124 receives the trans- portions with significant detail are not filtered at all. 

formed frame and reconstructs a reconstructed current frame Spatial filter 202 filters each component of the pixels of 

therefrom. Specifically, frame reconstructor 124 recon- the current frame independently of other components, 

structs motion -compensated macroblocks of the frame Briefly, color pixels can be represented using multiple 

received from transform decoder 122 by reference to a 60 components. For example, RGB pixels are represented by 

previously reconstructed frame stored in buffer 128. The red, green, and blue components. Similarly, YUV pixels are 

reconstructed current frame is stored in a buffer 126 and the represented by a luminance component (Y) and two chromi- 

reconstructed frame which is previously stored in buffer 126 nance components (U and V). The following example illus- 

is moved to buffer 128. Therefore buffer 128 stores a trates the advantage of filtering each component separately, 

reconstructed previous frame which is reconstructed from 65 Suppose the subject matter of the current frame includes a 

the previously encoded frame. Dequantizer 120, transform coarsely knit, monochromatic sweater. In YUV format, the 

decoder 122, and frame reconstructor 124 are conventional. sweater is represented with significant detail in the lumi- 
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nance component (the fine detail of the coarsely knit spatially filtered frame 312. Processing transfers from steps 

sweater) and significantly less detail in the chrominance 408 and 412, and from test step 410 if the variance of the 

components (since the sweater is monochromatic). Spatially subject block is greater than light threshold 304L, to next 

filtering each component of the current frame independently step 414 in which the next block of the current frame is 

allows a heavy filter to be applied to the chrominance 5 processed by comparator 306 (FIG. 3) according to steps 

components to effectively remove most, if not all, of the 404-412 (FIG. 4). When all blocks of the current frame have 

visible noise from the chrominance components while a been Processed according to the loop of steps 402 and 414, 

light filter or no filter is applied to the luminance component processing according to logic flow diagram 400 completes 

to preserve the detail of the luminance of the subject matter, an * s P atia ^ v filt 5^ frame * 12 < FIG f ; 3 > T /^ xnts h ' 

e.g., of the coarse knit of the sweater. u f^ eci mat f of ^ J^ll adaptlVe SpatiaI 

& . „ „ M „ M , . , filtering in the manner described above. 

Spaualfilter202includesablockparser3a2 whichparscs ^ fi q{ m ^ 4) ^ ^ 

the current frame into blocks of pixels as represented m the 3M ^ ^ m ^ ^ a ^ flQw d f 

loop defined by steps 402 (F G. 4) and 414. In one m * ^ sm ^ m ^ 3) ^ 

embedment, the blocks of pixels includes four rows and ^s^J^^^^^^,^^^ 

four columns of pixels The block size is chosen to be small H & ^ J q[ ^ ^ 

enough to allow effecuve adaptation of spatial filter 202 ^ £ Q determines the values of pixel 702 (FIG. 

(FIG. 3) yet large enough to prov.de effecUve filtering 7A)andadjacem ' pixels7 02N,702S ( 702E.aV.d702Wtothe 

Frames often uiclude areas of sigmficantly detaifcd subjec > J ^ ^ of 702 and 

matter adjacent to areas with little or no detail in the subject ^ median yalue ^ ^ of ^ jm ^ 

matter. Smaller block size* permits heavy filtering _of low- 20 5Q2 5) < rticlllarl 

detail areas very close to high-detad areas without unduly _ - „*L\„ • - ,,i „„. „,l,Ck £ t»«;^«ii» 

, . , - , j.-i tt effective in removing impulse noise which is typically 

blurring high-detail areas near low-detail areas. However, . , , , , * • i • * j • _ • j 

, , . *? B L L . , „ ... _ . ' introduced by electrical equipment used in capturing and 

blocksizes which are excessively smaU provide insufficien ^ ^ ^ si ^ nals> video cameras> ^ 

numbers of pixels for effective spaUa filtering. Four-pixel £ vid £ ^ j a is/re ^ rdeis> ter video 

by four-pixel blocte provide particularly go^results in one 25 ^ £F £ deficiencfcs of ^ 

embodiment .In the loop defined by steps 402 (FIG 4) and ^ si J filtcrin mechanisms to that ^ convenaonal 

4 , 14 * 6 l^t, y ; 4 b,0C r f fi P r ^ rf° 8 k typically asLme and address only Gaussian noise 

steps 404-412 by spatial filter 202 (FIG 3). During each J Mch ^ > D ^ than ^ ^ 

iteration of the loop, the particular block of pixels processed ^ A median g,,,,/^ „ ma ' t described above ^ not 

is sometunes referred to as the subject block. 30 particularly tf Kmovmg Gaussian noise bu , ^ 

Spatial filter 202 includes a comparator 306 determines pjmicularly effective at removing impulse noise, 

the degree of detail in the subject matter of the subject block. M the ed ^ o£ the subject block> Qne Qr 

Specifically, comparator 306 determines the variance, i e., ^ of me adjacen , ^ shown in nG ?A ^ nQt 

the squared error about the mean of the values of a available . For example( if pixe i 702 is in a comer of the 

parucuhr component of the p«els of the subject block in subject b , ock) , wo adjacent ixe , s afe unavailable . 

step 404 (FIG. 4). In test steps 406 and 410 comparator 306 Accordinglv> me new va]ue 0 f pixel 702 is the median of 

compares the variance determinec I u | step 404 (FIG. 4) to a ^ ixe , values> name , me value of ^ m ^ the 

predetermined heavy threshold 304H (FIG. 3) and to a ^ remaining adjaeent pixels . lf pixel 702 ^ ^ one ^ 

predetermined light threshold 3041, respectively. Heavy of the subjec , blockt onl four ^ yalues are availab , 6( 

threshold 304H represents a pixel block variance thresho d name , jxe , 702 and three remaining pixek . 

for low detail pixel blocks and indicator vanance threshold j^^^ me va i ue of pixel 702 is replicated to provide 

for heavy filtering. Light threshold 3ML represents a pixel , fifih M va , ue> ^ the median of , he flve ixe , valuM ^ 

block vanance threshold for medium detail pixel blocks and M ^ ^ value of ixe , 7ft2 

indicates a variance threshold for light filtering. In one Ar . .. .. ... .. cl , . , ™ r - 

... . . . . . ,„... . 6 . %^ . « After application of the media filter in step 502 (FIG. 5), 

embodiment heavy ^tr^eshold 304H has a value of twenty- £ (FIG. 3) applies a 9-pixel smoothing filter 

five and light threshold 304L has a value of one hundred. {Q ^ Qf ^ m J^ nd su £ ject block in st f p m 

Comparator 306 compares heavy threshold 304H to the (FIG. 5). In filtering pixel 702 (FIG. 7B), the mean of the 

variance of the subject block in test step 406. If the variance vahies of pixel 70 2 and of all adjacent pixels including 

of the subject block is less than heavy threshold 304H, 5Q pixels 702N> 702Sj 702E> 702w> 7 Q2nw, 702NE, 702SE, 

processing transfers to step 408 (FIG. 4) in which a heavy and 7 02SW is selected as the new value of pixel 702. When 

filter 310 filters the subject block to form a heavily filtered one or more ad j acem pixels arc unavaUable due to the 

block which is stored as part of a spatially filtered frame 312. location of pixel 702 along an edge or in a corner of the 

Step 408 (FIG. 4) is described below in greater detail. subject block, the new value of pixel 702 is the mean of the 

Conversely, if the variance of the subject block is not less 55 value of pixel 702 and of all available adjacent pixels. The 

than heavy threshold 304H (FIG. 3), processing transfers smoothing of step 504 (FIG. 5) helps remove any noise not 

from test step 406 (FIG. 4) to test step 410. In test step 410, already removed by the median filtering of step 502. Since 

comparator 306 (FIG. 3) compares the variance of the any impulse noise is effectively removed by the median 

subject block to light threshold 304L. If the variance of the filtering of step 502, none of the annoying and perceptible 

subject block is not greater than light threshold 304L, 60 artifacts resulting from smoothing of impulse noise are 

processing transfers to step 412 (FIG. 4) in which a light present. The performance and robustness of the smoothing 

filter 308 filters the subject block to form a lightly filtered filtering of step 504 are therefore significantly improved, 

block which is stored as part of spatially filtered frame 312. After step 504, processing according to logic flow diagram 

Step 412 (FIG. 4) is described below in greater detail. 408, and therefore step 408 (FIG. 4), completes. 

Conversely, if the variance of the subject block is greater 65 The light filtering of step 412 (FIG. 4) and light filter 308 

than light threshold 304L, no spatial filter is applied to the (FIG. 3) is shown in greater detail as logic flow diagram 412 

subject block and the subject block is stored as part of (FIG. 6). In step 602, light filter 308 (FIG. 3) applies a 
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5-pixel median filter to each pixel of the subject block in a mines the absolute value of the difference between corre- 
manner which is directly analogous to the median filter sponding pixels of the current and previous frames. From 
described above with respect to step 502 (FIG. 5). these determined absolute differences, absolute pixel differ- 
After application of the media filter in step 602 (FIG. 6), ence generator 118 determines the average absolute differ- 
light filter 310 (FIG. 3) applies a 5-pixel smoothing filter to 5 ence per pixel between the two frames The absolute pixel 
& , . . , * cj, , ... , . . rA i difference is a good indicator of overall differences between 
each pixel of the median-filtered subject block in step 604 r f 

^ ^ , x _ . . - M J _ A . . r r . two frames. In contrast, root-mean-square differences 

(FIG. 6) In filtering pixel 702 (FIG. 7A), the mean of the betwecn C o rreS ponding pixels of two frames exaggerates 

values of pixel 702 and of adjacent pixels including pixels h differences between only a few pixels of the frames. 

702N, 702S, 702E, and 702W is selected as the new value Z , ~ u - A A mn J . , . 

c • lw£ ™Z «• • * -i m Temporal filter 204 (FIG. 8) compares the received abso- 

of pixel 702. When one or more adjacent pixels are unavail- 30 , t . r , t v . r j4 . . • . . . 

L 7 , L 7 r • I j lute pixel difference to a predetermined threshold m test step 

able due to the location of pixel 702 along an edge or in a nM A v , . « . , flU . e « 

r 7_ fu , c ■ \%nr> * *t. 902 (FIG. 9) and bypasses all temporal filtering of spatially 

corner of the subject block, the new value of pixel 702 is the nu v , ' jr m r* ^\ a. i_ j ■ *• n 

r . _ J r . 7_ n - , - „ \, . . , filtered frame 312 (FIG. 3), thereby producing spatially 

mean ofthe value of pixe 702 ano^faU available adjacent ^ ^ frame 814, if m * rc ^ ive i 

pixels. The smoothing of step »6(M(nG^ helps remove any absolme ^ differcnce ^ man ^ determined 

noise not already removed by the median filtering of step 35 , r T u °\ , . . , . , , 

*™ ...... ' f 4 . n . JL threshold. In one embodiment, the predetermined threshold 

602 yet is lighter than the smoothmg of step 504 (FIG. 5) by . . , . , r 4l _.. 

i_ £i. /n^ i\ * *u u r *u represents a absolute pixel difference of thirty (30). 

heavy filter 310 (FIG. 3) to thereby preserve more of the * . . . . . . . • \ 

j ..V ' £i . '/ lL1 j- Conversely, if the received absolute pixel difference is not 

detail of the subject matter of the subject block. The median , .i_ t i_ j , ■ j .i_ L u l 

cn . - . t M /TTif^ *\ • #u ^ j greater than the predetermined threshold, processing by 

filtering of step 602 (FIG. 6) improves the performance and f . ftl4 ^fmr* e\. r * i . on^ ?■ u 

, # 6 v J u . r K t ctvA . 9n temporal filter 204 (FIG. 8) transfers to loop step 904 which, 

robustness of the smoothing filtering of step 604 u, an 20 £ i ; P loop in which 

analogous manner to that described above with respect to fa J f fe f M fe " f M m d £ m ^ 

steps 502 (FIG. 5) and 504 After step 604 (FIG. 6). ^ accordin For each of to 

processing according to logic flow diagram 412, and there- t f . „ % u . / . r 

r * ^*"» ^\ i . blocks of spatially filtered frame 312, processing transfers 

fore step 412 (FIG. 4), completes. from loop £ p ^ {Q ^ m itera S tioQ of the 

Thus, spatial filter 202 (FIG. 3) adaptively appUes heavy loop of steps m 916, the particular blocks of spatially 

spatial filtering to portions of the current frame which filtered frame 312 and the reconstructed previous frame 

represent subject matter of relatively little or no detail, light pr0 cessed according to steps 906-914 are sometimes 

spatial filtering to portions of the current frame which referred to herem as the subject current block and the subject 

represent subject matter of moderate detail, and no spatial previous block, respectively. When all blocks of spatially 

filtering to portions of the current frame which represent filtered frame 312 are processed, processing according to 

subject matter of relatively high detail. In addition, the logic flow diagram 90 0 completes. 

particular typ^s of spatial filtering applied by spatial filter Te } fiher 2Q4 ^ g) a ^ mc 

202 are specifical y designed to remove impulse noise which which tiaU fiUered frame 312 intQ bbcks of a 

is typically prevalent m digital video signals while conven- arlicuIaf size 16 fOWS aD(J 16 columns of ^ afld 

Uonal systems typically focus exclusively on Gaussian noise a bbck wMch the reconstructed vious 

and largely ignore non-Gaussian noise which is also fre- ffame iflt0 corr e Sponding , equa l-sized blocks. In one 

quently present in digital video signals. The current frame embodiment, the positions of corresponding blocks of the 

after adaptive spatial filtering, is stored as spatially filtered reconstructed previous frame are direcUy ^ alogous t0 the 

rame 40 corresponding positions of corresponding blocks of spatially 
Temporal Filter 204 fi l tered frame 312 . i n an alternative embodiment, the posi- 
Temporal filter 204 (FIG. 2) receives spatially filtered tions of corresponding blocks of the reconstructed previous 
frame 312 (FIG. 3) from spatial filter 202 and a recon- frame are motion compensated relative to the positions of 
structed previous frame from buffer 128 (FIG. 1) and corresponding blocks of spatially filtered frame 312. 
adaptively temporally filters spatially filtered frame 312 45 The illustrative block size described above, i.e., 16-by- 16 
(FIG. 3) using the reconstructed previous frame. Temporal pixels, is selected to effectively detect identical subject 
filter 204 is shown in greater detail in FIG. 8. Processing by matter between the reconstructed previous frame and spa- 
temporal filter 204 is illustrated by logic flow diagram 900 fii tered f rame 312. if the block size is too large, blocks 
(FIG. 9). which include both identical subject matter and changed 
Prior to temporally filtering any portion of spatially 50 subject matter are determined to include changed subject 
filtered frame 312, temporal filter 204 (FIG. 8) determines matter and the advantages of temporally filtering are not 
whether the differences between the current frame and the realized. Conversely, if the block size is too small, each 
previous frame are so great that temporal filtering is futile, block can frequently contain insufficient pixel data to accu- 
i.e., is likely to produce no appreciable improvements in rately determine whether a corresponding block of pixel data 
image quality. To make such a determination, temporal filter 55 represents the same subject matter. 
204 receives a measure of the differences between the i n step 906 (FIG. 9), a block comparator 806 (FIG. 8) of 
current frame and the previous frame from absolute pixel- temporal filter 204 measures a difference between the sub- 
difference generator 118 and compares the measure to a ject current block and the subject previous block to deter- 
predetermined threshold. Absolute pixel-difference genera- mine whether the subject current block and the subject 
tor 118 (FIG. 1) produces such a measurement in the form eo previous block represent the same subject matter. In one 
of an absolute pixel difference between the current and embodiment, block comparator 806 measures the difference 
previous frames. by computing a composite mean squared error of corre- 
An absolute pixel difference between two frames is the sponding pixels of the subject current and previous blocks, 
average of the absolute value of the difference of each pair The composite mean squared error is the average mean 
of corresponding pixels of the two frames. Absolute pixel 65 squared error of the various components of the pixels of the 
difference generator 118 retrieves the current and previous subject current and previous blocks. For example, if the 
frames from buffers 102 and 104, respectively, and deter- pixels of the subject current and previous blocks have three 
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components in the YUV format, the composite mean 
squared error is one third of the sum of (i) the mean squared 
error of the Y luminance components of corresponding 
pixels of the subject current and previous blocks, (ii) the 
mean squared error of the U chrominance components of 5 
corresponding pixels of the subject current and previous 
blocks, and (iii) the mean squared error of the V chromi- 
nance components of corresponding pixels of the subject 
current and previous blocks. The composite mean squared 
error provides a good indication as to whether the subject 10 
current and previous blocks represent the same subject 
matter since dissimilar subject matter can differ significantly 
in only one of the components of the pixels of the respective 
blocks. 

In test step 908 (FIG. 9), a pixel comparator 808 (FIG. 8) 15 
of temporal filter 204 receives the measured difference 
between the subject current and previous blocks from block 
comparator 806 and determines whether the measured dif- 
ference indicates that the subject current and previous blocks 
represent the same subject matter. In one embodiment, pixel 20 
comparator 808 makes such a determination by comparing 
the measured difference to a predetermined threshold, e.g., 
twenty-five (25), such that a measured difference greater 
than the predetermined threshold indicates that the subject 
current and previous frames do not represent the same 25 
subject matter. If, in test step 908 (FIG. 9), pixel comparator 
808 (FIG. 8) determines that the subject current and previous 
blocks represent different subject matter, processing trans- 
fers through next step 916 (FIG. 9) to loop step 904 in which 
the next block of spatially filtered frame 312 and the 30 
corresponding block of the reconstructed previous frame in 
a subsequent iteration of the loop of steps 904 and 916. 
Accordingly, steps 910-914, which are described below, are 
bypassed and no temporal filtering is applied to the subject 
current block. The subject current block is therefore 35 
included in filtered frame 814 (FIG. 8). 

Conversely, if pixel comparator 808 determines that the 
subject current and previous blocks represent the same 
subject matter, processing transfers from test step 908 (FIG. 
9) to loop step 910 and an adaptive temporal filter is applied 40 
to the subject current block. Loop step 910 and next step 914 
define a loop within which each pixel of the subject current 
block and the corresponding pixel of the subject previous 
block are processed according to step 912. During a par- 
ticular iteration of the loop of steps 910 and 914, the pixel 45 
of the subject current block processed by pixel comparator 
808 (FIG. 8) is sometimes referred to as the subject current 
pixel, and the corresponding pixel of the subject previous 
block is sometimes referred to as the subject previous pixel. 
In step 912 (FIG. 9), pixel comparator 808 (FIG. 8) adap- 50 
tively and temporally filters the subject current pixel in a 
manner illustrated by logic flow diagram 912 (FIG. 10) 
which shows test 912 (FIG. 9) in greater detail. In step 912, 
pixel comparator 808 (FIG. 8) filters each component of 
each pixel of the subject current block independently since 55 
pixel comparator 808 has already determined in test step 908 
(FIG. 9). 

In step 1002 (FIG. 10), pixel comparator 808 (FIG. 8) 
determines the pixel error between the subject current pixel 
and the subject previous pixel. Pixel comparator 808 mea- 60 
sures the pixel error between corresponding pixels of the 
subject current and previous blocks as the absolute differ- 
ence between corresponding components of the correspond- 
ing pixels. The pixel error indicates whether the subject 
current pixel is to be heavily, lightly, or not temporally 65 
filtered by pixel comparator 808. Such provides particularly 
good results in terms of video image clarity since blocks of 
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successive frame which represent the same subject matter 
can include slight differences in subject matter due to 
movement of the represented subject matter between frames. 
By determining the error of each pixel and adapting the filter 
accordingly prevents temporal blurring and still effectively 
removes noise from the digital video signal. 

In test step 1004 (FIG. 10), pixel comparator 808 (FIG. 8) 
compares the pixel error to a predetermined heavy threshold 
804H which has a value of five (5) in one embodiment. If the 
pixel error is not greater than heavy threshold 804H, pro- 
cessing transfers to step 1006 (FIG. 10) in which the subject 
current pixel is heavily temporally filtered using heavy niter 
812. In one embodiment, heavy filter 812 is a two-tap 
infinite impulse response (IIR) filter in which the subject 
previous pixel is weighted by a factor of 0.75 and the subject 
current pixel is weighted by a factor of 0.25 and the 
weighted pixels are summed to form a heavily filtered pixel 
which is stored in filtered frame 814 in place of the subject 
current pixel. 

If the pixel error is greater than heavy threshold 804H, 
processing transfers from test step 1004 FIG. 10) to test step 
1008. In test step 1008, pixel comparator 808 FIG. 8) 
compares the pixel difference to a predetermined light 
threshold 804L which has a value often (10) in one embodi- 
ment. If the pixel error is not greater than light threshold 
804L, processing transfers to step 1010 (FIG. 10) in which 
the subject current pixel is lightly temporally filtered using 
light filter 810. In one embodiment, light filter 810 is a 
two-tap inte impulse response (IIR) filter in which the 
subject previous pixel is weighted by a factor of 03 and the 
subject current pixel is weighted by a factor of 0.5 and the 
weighted pixels are summed to form a lightly filtered pixel 
which is stored in filtered frame 814 in place of the subject 
current pixel. 

After step 1006 or step 1010 or, if the pixel difference is 
greater than light threshold 804L, test step 1008, processing 
according to logic flow diagram 912, and therefore step 912 
(FIG. 9), completes. 

The IIR filters of heavy filter 812 (FIG. 8) and fight filter 
810 are particularly effective in removing noise from digital 
video signals. First, IIR filters provide smoothing over a 
potentially infinite number of previous frames since the 
subject previous pixel could have been filtered using a 
corresponding pixel of a frame preceding the reconstructed 
previous frame and that corresponding pixel could have 
been similarly temporally filtered and so on. Second, filters 
810 and 812 filter the subject current pixel more aggres- 
sively than do filters typically used in conventional systems. 
To avoid temporally blurring the digital video signal by such 
aggressive, heavy filtering, the determination of whether the 
subject current and previous blocks and the subject current 
and previous pixels represent the same subject matter by 
pixel comparator 808 must be accurate. The accuracy of 
such a determination is significantly improved by the 
removal of impulse noise and other noise by spatial filter 202 
(FIG. 2) prior to filtering by temporal filter 204. In addition, 
DC shifting noise, which is relatively common in digital 
video signals stored on old or low quality video tape or 
recorded to or retrieved from video tape using dirty or low 
quality tape heads and which is characterized by rapid 
lightening or darkening of successive frames of the motion 
video image, is effectively removed by temporal filter 204. 
Specifically, such lightening or darkening is sufficiently 
small between successive frames that heavy filter 812 (FIG. 
8) of temporal filter 204 effectively removes any DC shifting 
noise which is visible in the digital video signal on those 
portions of the digital video signal in which the same subject 
matter is represented in successive frames. 
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The result of the combination of spatial filter 202 (FIG. 2) 
and temporal filter 204 is that the motion video image is 
significantly improved without losing any significant sharp- 
ness or clarity. The relatively small block size used by spatial 
filter 202 along with the adaptive application of spatial 5 
filtering preserves edges in the represented subject matter of 
the digital video signal. At the same time, removal of 
impulse and other noise by application of heavy spatial 
filtering where appropriate and further removal of noise by 
relatively heavy temporal filtering where appropriate com- 10 
bine to improve clarity substantially. Temporal filter 204 can 
effectively remove noise from portions of the digital video 
signal which represents highly detailed subject matter pro- 
vided the same highly detailed subject matter is represented 
by previous frames. 15 

Such significantly improves the performance of compres- 
sion of the digital video signal by digital video signal 
encoder 100 (FIG. 1). Since the digital video signal has 
improved clarity, artifacts of video signal compression are 
reduced. In addition, the amount of data required to repre- 20 
sent the encoded digital video signal can be reduced 
significantly, e.g., by as much as one-half, without sacrific- 
ing motion video image quality. 

Inclusion of Video Signal Compressor in a Computer Sys- 
tem 25 

In one embodiment, video signal encoder 100 (FIG. 1) is 
implemented in a computer system. In particular, computer 
system 1100 (FIG. 1) includes a processor 1102 and memory 
1104 which is coupled to processor 1102 through an inter- 
connect 1106. Interconnect 1106 can be generally any inter- 30 
connect mechanism for computer system components and 
can be, e.g., a bus, a crossbar, a mesh, a torus, or a 
hypercube. Processor 102 fetches from memory 1104 com- 
puter instructions and executes the fetched computer instruc- 
tions. In addition, processor 1102 can fetch computer 35 
instructions through a computer network 1170 through net- 
work access circuitry 1160 such as a modem or ethernet 
network access circuitry. Processor 1102 also reads data 
from and writes data to memory 1104 and sends data and 
control signals through interconnect 1106 to one or more 40 
computer display devices 1120 and receives data and control 
signals through interconnect 1106 from one or more com- 
puter user input devices 1130 in accordance with fetched and 
executed computer instructions. 

Memory 1104 can include any type of computer memory 45 
and can include, without limitation, randomly accessible 
memory (RAM), read-only memory (ROM), and storage 
devices which include storage media such as magnetic 
and/or optical disks. Memory 1104 includes video signal 
encoder 100 which is all or part of a computer process which 50 
in turn executes within processor 1102 from memory 1104. 
A computer process is generally a collection of computer 
instructions and data which collectively define a task per- 
formed by computer system 1100. 

Each of computer display devices 1120 can be any type of 55 
computer display device including without limitation a 
printer, a cathode ray tube (CRT), a light-emitting diode 
(LED) display, or a liquid crystal display (LCD). Each of 
computer display devices 1120 receives from processor 1102 
control signals and data and, in response to such control 60 
signals, displays the received data. Computer display 
devices 1120, and the control thereof by processor 1102, are 
conventional. 

Each of user input devices 1130 can be any type of user 
input device including, without limitation, a keyboard, a 65 
numeric keypad, or a pointing device such as an electronic 
mouse, trackball, lightpen, touch-sensitive pad, digitizing 



tablet, thumb wheels, or joystick. Each of user input devices 
generates signals in response to physical manipulation by a 
user and transmits those signals through interconnect 1106 
to processor 1102. 

As described above, video signal encoder 100 executes 
within processor 1102 from memory 1104. Specifically, 
processor 1102 fetches computer instructions from video 
signal encoder 100 and executes those computer instruc- 
tions. Processor 1102, in executing video signal encoder 
100, reads frames from source video signal 1140, processes 
and encodes those frames in the manner described above, 
and stored the encoded frames in encoded video signal 1150. 

The above description is illustrative only and is not 
limiting. The present invention is limited only by the claims 
which follow. 

What is claimed is: 

1. A method for filtering at least a portion of a digital 
video signal, the method comprising: 

calculating a pixel variance for a block of source pixels 
from a current frame of the motion video image; 

applying a heavy spatial filter to the block of source pixels 
to form a block of spatially filtered pixels representing 
current subject matter of the current frame when the 
pixel variance is less than a first pixel variance thresh- 
old; 

applying a light spatial filter to the block of source pixels 
to form the block of spatially filtered pixels when the 
pixel variance is less than a second pixel variance 
threshold; 

calculating a pixel error between the block of spatially 

filtered pixels and a block of pixels of a previous frame 

of the. digital video signal; 
applying a heavy temporal filter to the block of spatially 

filtered pixels when the pixel error is less than a first 

pixel error threshold; and 
applying a light temporal filter to the block of spatially 

filtered pixels when the pixel error is less than a second 

pixel error threshold. 

2. The method of claim 1 wherein the step of applying a 
light spatial filter comprises 

applying a five pixel median filter and a five pixel 
smoothing filter to the block of source pixels, and 
further wherein the step of applying a heavy spatial 
filter comprises applying a five pixel median filter and 
a nine pixel smoothing filter to the block of source 
pixels. 

3. The method of claim 1 wherein the calculated pixel 
error is an absolute pixel difference. 

4. A computer readable medium useful in association with 
a computer which includes 

a processor and a memory, the computer readable medium 
including computer instructions which are configured 
to cause the computer to filter a digital video signal by 
performing the steps of: 

calculating a pixel variance for a block of source pixels 
from a current frame of the motion video image; 

applying a heavy spatial filter to the block of source 
pixels to form a block of spatially filtered pixels 
representing current subject matter of the current 
frame when the pixel variance is less than a first pixel 
variance threshold; 

applying a light spatial filter to the block of source 
pixels to form the block of spatially filtered pixels 
when the pixel variance is less than a second pixel 
variance threshold; 
calculating a pixel error between the block of spatially 

filtered pixels and a block of pixels of a previous frame 

of the digital video signal; 
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applying a heavy temporal filter to the block of spatially 

filtered pixels when the pixel error is less than a first 

pixel error threshold; and 
applying a light temporal filter to the block of spatially 

filtered pixels when the pixel error is less than a second 5 

pixel error threshold. 

5. The computer readable medium of claim 4 wherein the 
step of applying a light spatial filter comprises 

applying a five pixel median filter and a five pixel 
smoothing fitter to the block of source pixels, and 30 
further wherein the step of applying a heavy spatial 
filter comprises applying a five pixel median filter and 
a nine pixel smoothing filter to the block of source 
pixels. 

6. The computer readable medium of claim 4 wherein the 35 
calculated pixel error is an absolute pixel difference. 

7. A computer system comprising: 
a processor, 

a memory operatively coupled to the processor; and 2 0 

a digital video signal filter which executes in the proces- 
sor from the memory and which, when executed by the 
processor, causes the computer to filter a digital video 
signal by performing the steps of: 

calculating a pixel variance for a block of source pixels 25 
from a current frame of the motion video image; 

applying a heavy spatial filter to the block of source pixels 
to form a block of spatially filtered pixels representing 
current subject matter of the current frame when the 
pixel variance is less than a first pixel variance thresh- 30 
old; 

applying a light spatial filter to the block of source of 
pixels to form the block of spatially filtered pixels when 
the pixel variance is less than a second pixel variance 35 
threshold; 

calculating a pixel error between the block of spatially 

filtered pixels and a block of pixels of a previous frame 

of the digital video signal; 
applying a heavy temporal filter to the block of spatially 40 

filtered pixels when the pixel error is less than a first 

pixel error threshold; and 
applying a light temporal filter to the block of spatially 

filtered pixels when the pixel error is less than a second 

pixel error threshold. 
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8. The computer system of claim 7 wherein the step of 
applying a light spatial filter comprises 

applying a five pixel median filter and a five pixel 
smoothing filter to the block of source pixels, and 
further wherein the step of applying a heavy spatial 
filter comprises applying a five pixel median filter and 
a nine pixel smoothing filter to the block of source 
pixels. 

9. The computer system of claim 7 wherein the calculated 
pixel error is an absolute pixel difference. 

10. The computer system of claim 7 wherein the step of 
comparing comprises: 

measuring a difference between the block of spatially 
filtered pixels and the block of pixels of the previous 
frame; and 

determining whether the block of pixels of the previous 
frame represents the current subject matter by compari- 
son of the measured difference to a predetermined 
threshold. 

11. The method of claim 1, wherein the step of applying 
a heavy temporal filter includes applying a two-tap infinite 
impulse response filter in which the pixels of the previous 
frame are weighted by a factor of substantially equal to 0.75, 
and further wherein the step of applying a light temporal 
filter includes applying a two-tap infinite impulse response 
filter in which the pixels of the previous frame are weighted 
by a factor of substantially equal to 0.5. 

12. The computer readable medium of claim 4, wherein 
the step of applying a heavy temporal filter includes apply- 
ing a two-tap infinite impulse response filter in which the 
pixels of the previous frame are weighted by a factor of 
substantially equal to 0.75, and further wherein the step of 
applying a light temporal filter includes applying a two-tap 
infinite impulse response filter in which the pixels of the 
previous frame are weighted by a factor of substantially 
equal to 0.5. 

13. The computer system of claim 7, wherein the step of 
applying a heavy temporal filter includes applying a two-tap 
infinite impulse response filter in which the pixels of the 
previous frame are weighted by a factor of substantially 
equal to 0.75, and further wherein the step of applying a light 
temporal filter includes applying a two-tap infinite impulse 
response filter in which the pixels of the previous frame are 
weighted by a factor of substantially equal to 0.5. 



03/04/2004, EAST version: 1.4.1 



UNITED STATES PATENT AND TRADEMARK OFFICE 

CERTIFICATE OF CORRECTION 



PATENT NO. 
DATED 



: 6,281,942 Bl 
: August 28, 2001 



Page 1 of 1 



INVENTOR(S) :Wang 



It is certified that error appears in the above-identified patent and that said Letters Patent is 
hereby corrected as shown below: 



Column 12, 

Line 28, replace "inte" with -- infinite --. 
Column 15, 

Line 32, delete "of after "source". 



Signed and Sealed this 



Eleventh Day of February, 2003 




JAMES E. ROGAN 



Director of the United States Patent and Trademark Office 



03/04/2004, EAST Version: 1.4.1 



llllIlllllllllHill 

US006535254B1 

(12) United States Patent m Patent No.: US 6,535,254 Bi 

Olsson et al. (45) Date of Patent: Mar. 18, 2003 



(54) METHOD AND DEVICE FOR NOISE 
REDUCTION 

(75) Inventors: Kent Olsson, Sundsbruk (SE); Ting 
Ting Zhang, Sundvall (SE); Youshl 
Xu, Simdvall (SE); Roger Andersson, 
Sundvall (SE) 

(73) Assignee: Pinnacle Systems Inc., Mountain View, 
CA(US) 

( * ) Notice: Subject to any disclaimer, the term of this 
patent is extended or adjusted under 35 
U.S.C. 154(b) by 0 days. 

(21) Appl. No.: 09/056,056 

(22) Filed: Apr. 6, 1998 

Related U.S. Application Data 

(60) Provisional application No. 60/066,633, filed on Oct. 31, 
1997. 

(51) Int. CI. 7 H04N 5/21; H04N 3/208; 

G06K 9/00 

(52) U.S. CI 348/607; 348/625; 382/199 

(58) Field of Search 348/607, 625, 

348/620, 615, 616, 619; 382/199, 65, 194, 

210, 221 

(56) References Cited 

U.S. PATENT DOCUMENTS 

4,573,070 A • 2/1986 Cooper 358/36 

4,783,840 A * 11/1988 Song 382/54 



4,962,542 A ♦ 10/1990 Hees 358/54 

5,208,673 A 5/1993 Boyce 358/167 

5,294,984 A • 3/1994 Mori et al 348/551 

5,394,192 A 2/1995 HackeU 348/619 

5,404,178 A 4/1995 Kondo et al 348/618 

5,442,407 A * 8/1995 Iu 348/620 

5,491,519 A 2/1996 Kim 348/620 

5,585,859 A 12/1996 Ward et al 348/619 

5,598,226 A * 1/1997 Kokoram 348/607 

5,606,631 A • 2/1997 Weiss et al 382/275 

5,946,416 A • 8/1999 Akagi et al 382/194 

5,959,693 A • 9/1999 Wu et al 348/624 

FOREIGN PATENT DOCUMENTS 

WO WO 95/05717 2/1995 H04N/5/44 



* cited by examiner 

Primary Examiner — Andrew Faile 
Assistant Examiner — Reuben M. Brown 
(74) Attorney, Agent, or Firm — -Timothy Piatt 

(57) ABSTRACT 

A method of reducing noise in a video signal which includes 
a plurality of video frames being composed of a plurality of 
pixels, the method comprising the steps of: comparing video 
information contained in a current video frame and a plu- 
rality of temporally adjacent video frames; selecting from 
the current video frame and the adjacent video frames the 
video information that according to a predetermined condi- 
tion is likely to be correct for the current video frame; and 
finally assigning the selected video information to the cur- 
rent video frame to thereby produce a video frame wherein 
noise has been reduced. 

14 Claims, 5 Drawing Sheets 
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METHOD AND DEVICE FOR NOISE 
REDUCTION 

PRIORITY INFORMATION 

This application claims the benefit of provisional appli- 
cation No. 60/066,633, filed Oct. 31, 1997. This application 
claims Paris Convention priority under 35 U.S.C § 119 from 
Swedish patent application nos. SE 9703234-6, filed on Sep. 
8, 1997 and SE 9701251-2, filed on Apr. 4, 1997. A 
CD-ROM containing a computer program listing appendix 
has been submitted and is herein incorporated. The 
computer-readable compact disc (IBM-PC, 
MS-WINDOWS, 8751753.doc, 62 KB, Apr. 18, 2002, 09-37 
and P10352 as filed.doc, 108 KB, Apr. 18, 2002, 09.38 is 
submitted in duplicate herewith, containing the computer 
program code of Examples 1-5 of the specification. 

FIELD OF THE INVENTION 

The present invention relates generally to noise reduction 
in digitized pictures and specifically to the reduction of noise 
in a video signal. 

BACKGROUND OF THE INVENTION 

Noise reduction of a video signal is used to enhance the 
quality of images comprising the video signal and to prepare 
for an efficient compression of the video signal. Noise 
reduction is important in connection with compression of 
image information, because noise may significantly reduce 
the effectiveness of compression schemes, particularly 
frequency-domain compression schemes such as MPEG-2, 
In image compression technology there is typically a trade- 
off between compression and image quality; increased com- 
pression may tend to reduce image quality. It is not always 
easy to reconcile these differences so as to achieve high 
quality highly compressed images. Effective noise reduction 
in connection with compression of a video signal may well 
serve both purposes and produce enhanced images in addi- 
tion to a well-compressed video signal. 

The technical background of compression and different 
state of the art preprocessing and compression techniques 
are described for example in John Watkinson, Compression 
in Video & Audio, Focal Press 1995, ISBN 0 240 51394, 
which is incorporated by reference. 

A previously known video signal noise reduction system 
is described, for example, in U.S. Pat. No. 5361,105 to 
Siu-Leong Iu, the content of which is hereby incorporated by 
reference. In this system, image pixels are tracked across 
multiple frames and are averaged to produce respective 
noise-reduced pixel values. In video signals representing a 
sequence of images that may change from frame to frame, 
e.g. in motion pictures and television, this system seeks to 
reduce noise by estimating the magnitude and direction of 
inter-frame motion in a sequence of image frames. The 
movement of an image is estimated by first calculating a 
trajectory vector for a block of picture elements (pixels) by 
comparing preceding and succeeding frames. The trajectory 
vector is then used to reduce noise in the video signal by 
averaging each of a plurality of pixels corresponding to a 
moving block along an estimated trajectory. Tlie described 
method, however, does not eliminate noise to a satisfactorily 
sufficient degree. One drawback, for example, is that the 
trajectory estimation, which is a part of this noise reduction 
method, itself is sensitive to the noise. 

Another system is disclosed in U.S. Pat. No. 4,987,481 to 
Spears et al., which is hereby incorporated by reference. 



Spears shows an apparatus for selectively reducing noise by 
non-recursively averaging video information contained in a 
sequence of video frames when the video information is 
found to be impaired by noise. A drawback with this method 

5 is the consequence of the averaging between video frames. 
The video information of a noise-reduced frame portion is 
calculated and original video information is lost even if it is 
correct and unimpaired by noise. Furthermore, with the 
above procedure there is a high probability for different pixel 

10 values to appear or be introduced in sequence, which in its 
turn leads to poorer compression of the images in the video 
stream. 
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SUMMARY OF THE INVENTION 

The object of and the problem to be solved by the present 
invention is to reduce the noise in a video signal representing 
a sequence of video frames, in particular a video signal 
representing moving pictures. An aspect of this problem is 
to achieve an enhanced compression factor for a compres- 
sion scheme applied to the video signal, or stated differently, 
to reduce the bit rate of the video signal. A further aspect is 
to achieve an enhanced perceived image quality, preferably 
combined with an enhanced (or at least not reduced) com- 
pression factor. 

Accordingly, the result achieved by the invention can be 
considered a form of noise reduction, or alternatively, video 
signal optimization for compression. The method of the 
invention is operative to remove or "smooth out" minor 
differences in a video signal, ordinarily imperceptible or 
only marginally perceptible to the human eye, so that certain 
types of data compression can be performed on the video 
data more efficiently. In particular, the well-known MPEG 
compression scheme uses coefficients in the frequency 
domain to represent small eight-by-eight pixel regions of an 
image. Given a constant target bit rate, eliminating many of 
the high-frequency components devoted primarily to 
"noise" will leave more bandwidth for the low-frequency 
components of the compressed MPEG video stream, thereby 
leading to a possible increase in perceived image quality. 

The invention is based on the discovery that there is a 
relationship or correlation between video information of 
frame portions within an observed video frame (i.e. a spatial 
correlation) on one hand, and between video information of 
frame portions from a sequence of adjacent observed video 
frames (i.e. a correlation along the time axis of the frame 
sequence) on the other hand. The correlation is strong in 
sequences of frame portions where there is no local scene 
change or movement in the image. A local scene change in 
this context means that the video information changes 
significantly between sequential corresponding frame 
portions, i.e. over time. 

For a two-dimensional video signal, the correlation analy- 
sis is carried out in three dimensions, i.e. with respect to the 
surrounding pixels that are adjacent to the current pixel in 
the two-dimensional spatial domain and in the time domain, 
respectively. In order to distinguish between a pixel that 
represents a local scene change (e.g. an edge of a moving 
object within the image or a cut to another scene) and a 
random pixel or noise spike, the correlation between the 
current pixel under consideration and its surrounding pixels 
is analyzed. Noise on the current pixel, if any, is suppressed 
on the basis of the correlation analysis. If there is a weak 
correlation, a local scene change is assumed to take place 
between compared frame portions, and no attempt is made 
to reduce noise on this pixel. If, on the other hand, a strong 
correlation is found between compared frame portions, 
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typically two frame portions, those frame portions qualify 
for further processing in a selecting step, with possible 
subsequent noise reduction. 

According to the invention, the noise on a pixel is 
preferably replaced by a maximum likelihood signal based 
on correlated pixels. Stated differently, from a subset of the 
values of correlated pixels, the pixel that has a value which 
is most likely to have a similar predecessor or successor is 
selected to replace the current pixel. Each video frame is 
processed under consideration of a sequence of video frames 
temporally adjacent to the current video frame. The tempo- 
rally adjacent frames may in various embodiments be ahead 
of, come after, or surround the current video frame. In a 
preferred embodiment, the processing is performed for each 
pixel of a current video frame by observing a frame portion 
consisting of the current pixel and a number of surrounding, 
spatially adjacent pixels. Such a frame portion is called a 
slice S. Each current slice, or more specifically the video 
information of the slice, is compared to equal sized spatially 
corresponding slices of the adjacent frames. Such a set of 
temporally consecutive slices can be called a tube T. The 
tube T is analyzed in order to select the slices adjacent to the 
current slice that have a particularly strong correlation, i.e. 
the consecutive slices wherein there is no local scene 
change. 

Having selected a set of temporally consecutive slices that 
do not have a local scene change, the current pixel is 
compared to the spatially corresponding but temporally 
adjacent pixels of the selected slices (a subset of the original 
slices) and the pixel that is most likely to have a preceding 
or subsequent pixel having the same value or video infor- 
mation content is selected as a new current pixel. In a 
different wording, an extreme pixel value is sorted out and 
is replaced by a pixel selected from a set of adjacent pixels 
and being judged to have a better correlation to the sur- 
rounding pixels. The selected pixel is then assigned to the 
current pixel. 

In one embodiment, the pixel judged to have a desired 
correlation, and which is assigned to the current pixel, is a 
pixel having the median luminance value of the pixels in the 
selected slices forming a tube of slices that is defined by 
two" local scene changes (i.e. "before" and "after the current 
scene). 

In one embodiment, an entirely new sequence of video 
frames is produced in the noise reduction process, whereas 
in another embodiment, the current video frame is replaced 
by the new frame which thus may contribute in the analysis 
and processing of the next current video frame. 

When the invention is applied as a pre-processing tool for 
compression of an image signal, an object of the invention 
is to produce sequences of pixels having the same value, but 
without loss of significant image information. In advanta- 
geous cases, this leads to a number of subsequent similar or 
identical blocks of (e.g. 8x8) pixels which, when encoded in 
a subsequent coding stage, are highly compressed. 

In a further embodiment, account is also taken of the fact 
that the human eye has differing sensitivity to light and dark 
and to the different colors red, green and blue. Noise- 
impaired pixels are then discriminated with regard to a 
dynamically computed noise threshold value, such that the 
precision in noise and scene change detection is adapted to 
the characteristics of human vision. For example, detailed 
changes are hard to detect or are even undetectable in dark 
and in very bright areas. By taking the human eye sensitivity 
into consideration while computing the noise threshold 
value, the bit rate may be reduced further in regions of 
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frequency spectra, or color space where human is less 
sensitive to small differences. An image can be achieved 
which is subjectively perceived by the eye as enhanced, even 
though video information may actually be missing or noise 
5 actually is left in parts of the picture. 

In other embodiments, a predetermined or fixed threshold 
value is selected and used for a chosen selected range or for 
the whole frequency range of the picture. 

Further advantages and details of the invention will be 
10 seen from the following description of an embodiment of the 
invention with the aid of the accompanying drawings and in 
connection with the independent and dependent claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 

15 

FIG. 1 shows a sequence of video frames illustrating the 
tube and slice concepts of the invention; 

FIG. 2 depicts a sequence of video frames, a trajectory, 
and frame portions as used in a prior art noise-reduction 
20 scheme; 

FIG. 3 shows an example of a slice used in one embodi- 
ment of the invention; 

FIG. 4 shows a general view of an embodiment of the 
inventive noise reduction system and method, including 
storage for seven consecutive frames; 

FIG. 5 shows an overview of a noise reduction system 
capable of performing the method of the invention; 

FIG. 6 sets forth the functional operations performed by 
30 one embodiment of the noise reduction system set forth as 
a portion of FIG. 5; and 

FIG. 7 sets forth the functional operations performed by 
an alternative embodiment of the noise reduction system set 
forth as a portion of FIG. 5. 

35 

DETAILED DESCRIPTION OF THE 
INVENTION 

Referring initially to FIGS. 1-2, a video signal or an 

40 image signal comprises a sequence of video frames F, 
wherein image information is stored in a spatial domain. In 
moving images, the video information basically changes 
from frame to frame along a time axis t. In the inventive 
method of FIG. 1, a video frame of digitized video infor- 

45 mation represented in a plurality of pixels P is processed 
portion by portion, where a frame portion typically would 
comprise at least o ne currently processed pixe l and a number 
o f adjace nt pixels. 
In a prior art system, as shown in FIG. 2, a trajectory Traj 

50 of frame portions FP is calculated, and serves as a descrip- 
tion of the path of a moving image object that is moving 
across a set of frames as a function of time. Such a trajectory 
is estimated in U.S. Pat. No. 5361,105. Noise reduction is 
carried out by first tracking moving image pixels P to 

55 estimate their trajectory and then producing a supposedly 
noise-reduced pixel value for the tracked pixel. This model 
can be less efficient than desired, because the process of 
estimating moving images involves complex calculations 
which can be slow, and the model is itself sensitive to noise. 

6 o The invention carries out noise reduction processing without 
this kind of motion estimation or trajectory tracking. 

In the invention, a number of frame portions called slices 
S (FIG. 1), preferably comprising a number of pixels P 
having corresponding spatial positions in a window W, a 

65 sequence of temporally adjacent frames F, are observed. 
Unlike the trajectory model, such a set of slices (called a 
tube T), is parallel to the time axis t. Each pixel is associated 
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with some kind of color data, e.g. for each of the primary 
colors red R, green G and blue B, also referred to as RGB. 
Naturally, other image data or color representations are also 
possible within the inventive concept. 

For the purpose of the description of the invention, the 5 
following expressions and corresponding mathematical 
notation are used. The meaning of the expressions is also 
clarified by way of the figures. 
Frame: F(t)o{R(t), G(t), B(t)} 

is a frame signal matrix of RGB values at the t:th frame. 10 
Pixel: P(ij,t)-{R(io,t), G(i,j,t), B(ij,t)}eF(t) 
is a picture element in frame F(t), with the position coordi- 
nates ij In the t:th frame. 
Luminance: Y(ij,t) 

is the luminance of the pixel P(i j,t). The luminance Y can be is 
calculated by means well-known in the art, for example, 
from the RGB values of the pixel P. 
Slice: S(ij,t) 

is a collection of pixels forming a subset of the t:th frame. 
It forms a designed spatial window or 'slice*. In preferred 20 
embodiments, a slice includes the center pixel P(iJ,t) and 
some surrounding pixels P(i'j',t). The notation i'j',! 1 means 
a position or time reference being adjacent (in time, space, 
or both) to a currently processed pixel. FIG. 3 shows an . 
example of a slice in the form of a cross having a current 25 
pixel P ( y in the center position. 
Window: W(t)-{F(x)|t-d^x^t+d} 

is a set of consecutive frames, where d is a positive integer. 
F(t) is the center frame and (2d+l) is the size of a sliding 
"time- window" of consecutive frames. The term x represents 30 
a time, as defined by the expression, and generally indicates 
a range of frames (F) from time t-d to time t+d. 
Tube: T(ij,t)-{S(ij,t)|t^d^x^t4d} 
is a collection of slices, or a three dimensional 'time-tube' of 
pixels. It typically consists of (2d+l) consecutive "slices", 35 
one from each of the consecutive frames belonging to the 
time-window W(t). In one embodiment, a tube typically 
includes the center "slice" S(ij,t) and its adjacent-in-time 2d 
"slices" S(ij,f). 

As will be discussed in further detail below, in one 40 
embodiment of the invention each pixel is processed gen- 
erally as follows. First, a number of temporally adjacent 
pixels that have a strong correlation (i.e. that are not 
involved in a local scene change) is determined. Second, 
from the set of strongly correlated temporally adjacent 45 
pixels, a pixel value that is most likely to have a predecessor 
or successor of equal or similar value is selected. Third, the 
currently processed pixel is replaced with the selected pixel. 

The process of identifying the correlated pixels, i.e. the 
scene change analysis, is in one embodiment carried out by 50 
observing a slice containing the current pixel and spatially 
adjacent pixels, i.e. a slice S(i,j,t), as well as a number of 
spatially corresponding but temporally adjacent slices S(i j, 
t'). The slices may for example consist of a center pixel P(i j) 
and a selected number of the closest adjacent pixels P(ij) 55 
forming a shape, e.g. a square, a cross or any other selected 
shape of pixels. A geometric shape is preferred. In one 
embodiment of the invention (illustrated in FIG. 3), a slice 
S comprises five pixels in a cross shape. A center pixel 310, 
represented by P iy , is flanked horizontally by a left pixel 312 60 
(at P v-1 ) and a right pixel 314 (at P (J+1 ), and is flanked 
vertically by a top pixel 316 (at P,. w ) and a bottom pixel 318 
(at P (+1 j). The cross-shaped configuration of FIG. 3 has been 
found to provide good results while minimizing computa- 
tional complexity. 65 

A more detailed exemplifying embodiment of the inven- 
tion comprises the following features and steps of: 



(a) Receiving as an input a video signal which includes a 
plurality of video frames of digitized video information 
represented in a plurality of pixels. 

(b) Storing a^jr^Lyide.o_ Jrame a nd a plurality of 
temporally adjacent video frames, typically an uneven 
number of video frames, e.g. seven frames, surrounding 
a centrally positioned and currently processed frame. 

(c) Selecting from the current video frame a current frame 
portion, i.e. a slice S(ij,t) comprising a current pixel 
P(i,j) and at least one pixel spatially adjacent to the 
current pixel. In a preferred embodiment, cross-shaped 
slices (as shown in FIG. 3) are used, assuming there is 
at least some degree of spatial correlation between the 
pixels of the slice. 

(d) Selecting from the temporally adjacent video frames 
temporally adjacent frame portions, i.e. slices, com- 
prising a pixel or pixels that spatially corresponds to the 
pixels of the current frame portion. In other words, a 
collection of consecutive slices S(ij,t') making up a 
tube T (FIG. 1). 

(e) Determining the luminance value of each pixel in the 
current slice and the adjacent slices, i.e. in all slices of 
the selected tube. 

(f) Determining the average luminance value of the cur- 
rent slice and the temporally adjacent slices, i.e. by 
averaging the luminance values of the pixels comprised 
in the frame portions. 

The foregoing steps can be seen as a preparation for 
making a subsequent correlation analysis, and more specifi- 
cally to determine whether a local scene change occurs 
between the slices of the tube. This is done by comparing the 
current frame portion and each of the adjacent frame por- 
tions by: 

(g) Determining and comparing to a first threshold value 
the absolute value of the difference between the aver- 
age luminance of the current slice and the average 
luminance of the adjacent slice. If the absolute value of 
the luminance difference exceeds the first threshold 
value, there is a local scene change in the video 
information between the current slice and the adjacent 
slice. In other words, if there is a large difference in 
luminance between adjacent frames, there is considered 
to be a local scene change between the current slice and 
an adjacent slice under consideration. 

If the luminance difference is small, a further correlation 
test as follows: 

(h) Determining and comparing to a second threshold 
value the absolute values of the difference between the 
values of a color component attribute (for example, red, 
green, or blue) of the current pixel and the correspond- 
ing pixel of the adjacent frame portion. If the absolute 
value of the color component value difference exceeds 
the second threshold value, there is a local scene 
change in the video information between the current 
slice and the adjacent slice. This second test involving 
color attributes is used to detect local scene changes 
where the luminance is more or less continuous despite 
a color change, which in effect is a local scene change. 
For example, where a scene change takes place, an 
object of one luminance may be replaced by an object 
having a similar luminance but a different color. In this 
case, the scene change should still be detected. 

If these threshold tests are passed, it is determined that no 
local scene change occurs between the current slice and the 
adjacent slice, and the next pair of slices are analyzed until 
the whole tube has been processed. Thereafter, zero or an 
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odd number of adjacent slices or frame portions is selected ences in the sensitivity of the human eye to different 

from the tube which are considered to involve no local scene wavelengths of the light. The fact that amplitude differences 

change in relation to the current frame portion. This can also are compared to a minimum threshold value entails a 

be expressed as selecting the largest possible sub-tube or discrimination between noise-impaired pixels and pixels 

sub-set of slices that does not involve a local scene change. 5 belonging to a moving part of the image represented by the 

For example, if the input tube comprises seven slices, a frame, whereby small changes in amplitude are reduced or 

subset consisting of three, five, or seven slices can be eliminated. The choice of threshold level is dependent on the 

selected, when as in this case a number of slices symmetri- application of the invention, and it is within the scope of the 

cally surrounding the current slice are analyzed. invention to select or adapt threshold to other parameters 

Thereafter, the actual noise reduction steps take place. 10 than human eye sensitivity. 

This is preferably done by selecting the video information (2) When all pixels P in F^, have been processed accord- 

from a pixel that is most likely to have similar predecessors mg to step (1) above, frame F^, is transferred to an output, 

and successors, for example by: Using the exemplifying shift registers (the memory M of 

(i) Comparing the luminance values of the current pixel FIG. 4), the other N frames F n to F^ are shifted to the left, 

and the corresponding pixels in the selected adjacent 15 so that F a is copied to F^, F^ is copied to F n and so on. A 

frame portions and selecting the video information new F^ is taken into the memory M from an input frame 

attributes of the pixel having the median of the lumi- stream of video signal. If there are no more frames at the 

nance values. input, the new F^ inherits the values of the old F^. 

(j) Assigning the selected video information attributes to Thereafter the next frame is processed according to step (1) 

the current pixel to thereby produce a video frame 20 above. 

having reduced noise. The noise reduction can be A further embodiment of the invention described in a 

performed in different applications and embodiments mathematical notation may comprise the following steps for 

either to enhance image quality or to achieve a required producing a noise filtered output pixel P^/ij,!): 

compression rate in an encoded image signal. In advan- Step 1: 

tageous cases, the two purposes may be combined. By 25 Calculating a luminance matrix 
selecting the median value of the set of temporally 

adjacent pixels, anomalies such as noise spikes are { Y ta (i'j',t')|V(i'j , ,t'Kr} 

filtered out. It is more likely that a current pixel taking 

on the attributes of pixel having the median attribute or step 2: 

attributes of a sequence of correlated pixels will be 30 Calculating the matrix of an indicate function I for the 

perceived to have a correct value, that is, it is more luminance differences between adjacent pixels, where 
likely to match preceding or succeeding pixels. 

FIG. 4 shows one embodiment of the invention in which 0|V(i'j' f)eT} 

a memory M, for example a shift register, is provided to hold 

a number N+l of first frames F^-F^ of an input video 35 anc j wnere 

signal. In the illustrated example, seven frames F^ to F rt are 

shown (as frame storage blocks 410-422). As illustrated, F^ ( 0 if |r (/ y r) - y- (f / OU £ 

isjhe currently processed frame and F^-F^ the six follow- /(f. /,/*)» j 

ing framesTwhich are also called 1 look-ahead' frames. Each w 

pixel P stored in each of the frames F^ to F^ contains color 40 

information, e.g. RGB attribute values. In another embodi- and E fe lhe value of a designed threshold function f rtr of the 

ment of the invention, the.me mory M holds a numb erof selected parameter components, in this example the color 

prceeaen^faunesi^and subsegue nTframes h^ maddi- components, of the input pixel, 
tipnjo-thecurrent frame F^. All ofthe tramestorage blocks 

410-422 provide frame data to a noise reduction processor 45 E-f /V {P if (i'j;t>w i(t (t)} 
424, which performs the steps set forth herein to accomplish 

noise reduction. As shown in FIG. 4, seven frames are used if j» f i*).i f the pixel is caUed an unequal pixel, 

as input to create a single output frame 426. In a preferred , n one cmbodimcni of the invention, the threshold func- 

embodiment, a maximum "tube depth of seven frames has tion f is a conslanl| and £ ^ consequently ^ a constant . 

been found to balance the need for computational efficiency 50 Howevcff ^ an ^^vc embodiment ofthe invention, the 

with the need to provide a sufficient number of frames to thresh old function L. . can be related to the color or other 

accomplish meaningful noise reduction. aUribllte-oTlhe"^ in order to 

Another embodiment of the invention comprises the fol- tallonbTThrShold to the sensitivity of the human eye at 

lowing steps, which are repeated and carried out for each different colors 

frame in a stream of frames of an input video signal. 55 « - 

(1) The c olor compon ent attributes of each pixel in the G £ ' . 
frame, inlhl^n^ife red, green and blue, .^ lc " labDg lhe number of uncc J ual P ixek lD each slice 
of a_c,urrontly_proccsscd.frame are individually compared to ^ or 
preferably all of the N succeeding values of the correspond- 
ing pixel in the subsequent look-ahead frames. If the dif- 60 
ferencesjn amplitude between a currently processed pixel 
P^ and a number of, and preferably all of, its succeeding ^ T '~ ^ IJ ' T ' 
pixels are below a_predeter mined thresh oldjevel, the value . . v 
of the nexTyixelT^iscopied to P^ otherwise P^, is left whcre {l J 
unchanged. 65 ^ te P ^ : 

Different threshold levels are specified and used for the Calculating the number of unequal pixels in the three 

different pixel components, thus taking account of differ- dimensional tube T(i,j,t) 



f-rf+l, . . . , t+4 separately 
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Step 5: 

Determining the output pixel 



{PiaV* 7. ') if hj s An*, and V j s i*,^) 

/. ') otherwise 



where X, Mfctf and pt slice are designed thresholds and 

PJij>0-P*i(ij.O for some t' at which Y^iAfi is the 
median value of the set {Y to (ij,T)|t-d^x^t+d}. In other 1S 
words, if there are a large number of unequal pixels in the 
tube or in each slice, then the input pixel is assumed to be 
noise, and is replaced with a pixel having median luminance. 

When all the output pixels P^/iJ »0*P«iX0 for different 
indexes (ij) are obtained, a noise reduced frame F OMl (t) is 20 
completed. Then, a new frame is input and processed 
according to the above described steps. 

FIG. 5 shows an overview of a system capable of accom- 
plishing noise reduction on an analog video signal The 
system accepts an analog video signal V, which is first 25 
sampled by a "frame grabber" or digital sampler 510 to 
convert the video signal V into the digital domain for further 
processing. The sampler 510 passes a digital signal repre- 
sentative of the analog video signal V to an input format 
converter 512, which identifies and separates the red, green 30 
and blue (RGB) components of each pixel of a sampled 
frame. Each RGB-separated frame is then passed to a noise 
reduction system 514, which processes the frames as 
described above to reduce the noise in the video signal V. As 
further described above, the noise reduction system 514 35 
includes both frame memory 51 6 for storing a sequence of 
frames to be processed and pixel memory 518 for storing a 
set of pixels to be operated upon. After noise reduction, an 
output frame is fed from the noise reduction system 514 
to an output format converter 520, which in one embodiment 40 
is provided with a digital frame output 522 for outputting a 
digital video output signal. In an alternative embodiment, 
the output format converter 520 is also provided with a 
digital to analog converter 524 and an analog video output 
526 for outputting a modified analog video signal. In one 45 
embodiment, the output format converter 520 is provided 
with both kinds of outputs. 

The invention can be realized by a hardware implemen- 
tation or by a software implementation executed on an 
information processing apparatus, such as a work station, 50 
personal computer, signal processor or the like. 

One embodiment of the noise reduction system 514 of 
FIG. 5 can be described in more detail with reference to the 
block diagram of FIG. 6. Frames received from the input 
format converter 512 (FIG. 5) are first sent to the frame 55 
memory 516 and the pixel memory 518. Both the frame 
memory 516 and the pixel memory 518 provide data as 
necessary in the subsequent functional steps set forth below. 
The noise reduction system 514 calculates a luminance 
matrix Y for a pixel matrix, namely the tube T, stored in the 60 
frame memory 516 and the pixel memory 518 (step 610). 
The noise reduction system 414 then calculates the lumi- 
nance difference indicate function I (step 612) based on an 
input pixel (from the pixel memory 518) and a predeter- 
mined and stored threshold function 614 (which in one 65 
embodiment of the invention is a constant). The system 514 
then calculates the number of unequal pixels in a slice (step 



616), as described above, and further calculates the number 
of unequal pixels in the tube T (step 618). Based on these 
calculations, the system 514 identifies a pixel having a 
median luminance from among the slices in the tube T (step 
620) and replaces the input pixel with the identified median 
pixel (step 622). This process is repeated for all of the input 
pixels in the frame. 

An alternative embodiment of the noise reduction system 
514 of FIG. 5 is shown in the block diagram of FIG. 7. 
Again, frames received from the input format converter 512 
(FIG. 5) are first sent to the frame memory 516 the pixel 
memory 518. Both the frame memory 516 and the pixel 
memory 518 both provide data as necessary to the subse- 
quent functional steps set forth below. The noise reduction 
system 514 calculates a luminance matrix Y for a pixel 
matrix, namely the tube T, stored in the frame memory 516 
and the pixel memory 518 (step 710). Th e noise reductio n 
system 514 then calculates an average luminan ce,for-each 
slice7-a nd "de termines wHelher the average luminance dis- 
closes the existence of a local scene change (step 712). If 
not, the color of the input pixel (from the pixel memory 518) 
is compared to the corresponding pixel in another slice (step 
714). If the tests indicate that there is no local scene change, 
then the system 514 identifies a pixel having a median 
luminance from among the slices in the tube T (step 716) and 
replaces the input pixel with the identified pixel (step 718). 
If there is a local scene change, then the identified median 
pixel (step 716) is the input pixel. This process is repeated 
for all of the input pixels in the frame. 

Five exemplary computer software routines are provided 
as Examples 1 through 5. These Examples illustrate a 
preferred embodiment of the noise reduction invention. 

Example 1 illustrates, in pseudocode, a modified version 
of an algorithm according to FIG. 7. As described above and 
in conjunction with FIG. 7, the Example 1 embodiment uses 
the average luminance of a slice to make noise reduction 
decisions, whereas the FIG. 6 embodiment counts "unequal 
pixels." This luminance implementation is less computa- 
tionally intensive than the implementation that counts pix- 
els. 

Example 2, the TNR.CPP file, sets forth an implementa- 
tion of the pseudocode of Example 1 in the C++ program- 
ming language. The pixel_sorter function is a set of routines 
capable of sorting, from lowest to highest, pixel attribute 
values in groups of three, five, or seven. Given the sorted 
pixel attribute values, the median_of function returns the 
attributes of the pixel having the median attributes from 
among the three, five, or seven returned by the pixel_sorter 
function. The same_scene function, as given, considers 
luminance only. This can produce some visible effects when 
luminance is unchanged, but color has changed (e.g. human 
skin on one frame followed by a blue object with the same 
luminance on the next frame). These effects can be limited 
by also considering color information to determine if the 
slice has changed (i.e., whether a scene change has 
occurred). The TNR function at the end of this software 
routine performs the entire algorithm, as described generally 
above and in Example 1. 

Example 3 includes the C++ header file information, 
including data structures and function prototypes, used in 
the TNR.CPP file set forth in Example 2. 

Example 4, TNR_UTIL.CPP, includes several C++ sub- 
routines used in the TNR.CPP file set forth in Example 2. 
The function luma_computer computes the average of a 
slice's luminances. This is a group of routines, each of which 
is tailored for a specific purpose. For example, within a 
single slice, a pixel might have 2, 3 or 4 neighbors, and 
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luminance cannot be derived from a non-existent pixel. The 
TNR_firame function computes the average luminance for 
the slice centered around each pixel. The function TNR_ 
framestore sets up a sequence (or "window") of frames for 
processing by the invention. If there are too few frames in 5 
the sequence, the window is "padded" by repeating the first 
frame at the beginning of the window or the last frame at the 
end of the window, as necessary. 

Example 5 is the header file information used by the 
TNR_UT1L.CPP file of Example 4. Specifically, TNR_ 10 
UTIL.CPP sets forth various data structures used for frame 
and pixel storage by the method of the invention as imple- 
mented in these Examples. 

The invention is described in connection which certain 
embodiments and examples which are not limiting. It will be is 
appreciated that the invention may be be adapted or modi- 
fied to suit different specific applications within the scope of 
the appended claims. 

What is claimed is: 

1. The method for reducing noise in a video signal that 20 
includes a plurality of video frames of digitized video 
information represented in a plurality of pixels, the method 
comprising the steps of: 

storing a current video frame and a plurality of temporally 
adjacent video frames; 25 

selecting from the current video frame a current frame 
portion comprising a current pixel and at least one pixel 
spatially adjacent to the current pixel; 

selecting from the temporally adjacent video frames tem- 3Q 
porally adjacent frame portions comprising a pixel or 
pixels that spatially corresponds to the pixels of the 
current frame portion; 

determining the luminance value of each pixel in said 
current frame portion and said adjacent frame portions; 35 

determining the average luminance values of said current 
frame portion and said spatially adjacent frame por- 
tions; 

determining and comparing to a first threshold value the 
absolute value of the difference between the average 40 
luminance of said current frame portions and the aver- 
age luminance of said adjacent frame portion; 

if the absolute value of the luminance difference exceeds 
said first threshold value, assuming there is a local 
scene change in the video information between said 45 
current frame portion and said adjacent frame portion; 
else 

determining and comparing to a second threshold value 
the absolute values of the difference between the values 
of a color component of said current pixel and the 50 
corresponding pixel of said adjacent frame portion; 

if the absolute value of the color component value dif- 
ference exceeds said second threshold value, assuming 
there is a local scene change in the video information $$ 
between said current frame portion and said adjacent 
frame portion; else 

assuming that there is no local scene change between said 
current frame portion and said adjacent frame portion; 

selecting an even number exceeding or equaling zero of $o 
adjacent frame portions considered to involve no local 
scene change in relation to said current frame portion; 

selecting the video information from a pixel that is 
probable to have equal predecessor and/or successors 
by: 65 
comparing the luminance values of said current pixel 
and the corresponding pixels in said selected adja- 
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cent frame portions and selecting the video informa- 
tion of the pixel having the median value of said 
luminance values; 
assigning said selected video information to the current 
pixel to thereby produce a video frame wherein noise 
has been reduced. 

2. The method according to claim 1, wherein the video 
information of the current video frame is replaced by the 
selected video information. 

3. The method according to claim 1, wherein the selected 
video information is used to produce a new current video 
frame. 

4. The method according to claim 1, wherein said plurality 
of temporally adjacent video frames are a predetermined 
number of video frames symmetrically preceding and/or 
succeeding a current video frame. 

5. An apparatus for reducing noise in a video signal that 
includes a plurality of video frames, the apparatus compris- 
ing: 

an input (22) for receiving an input frame of a set of input 
frames (Fin); 

memory means (2) for storing one or more input frames 
(Fin); 

memory means (3) for storing one or more pixels of a 
frame (F) to be processed; 

means (24) for calculating a luminance matrix (Y) for a 
pixel matrix stored in the memory means (3); 

means (26) for calculating a matrix (I) of a luminance 
difference indicate function dependent on a predeter- 
mined and stored threshold function (27); 

means (28) for calculating the number of unequal pixels 
in a slice; 

means (30) for calculating the number of unequal pixels 
in a tube (T); 

means (32) for determining an output pixel (Pout); and 
an output (34) for outputting one or more noise reduced 
frames (Fout). 

6. The apparatus according to claim 5, wherein said 
threshold function (27) is a constant value. 

7. The apparatus according to claim 5, wherein said 
threshold function (27) is adapted to the sensitivity of a 
human eye for different colors. 

8. The apparatus according to claim 5, wherein said 
threshold function (27) is dynamically computed, adapted to 
the characteristics of the human vision regarding the capa- 
bility to detect detailed changes in very dark and very bright 
areas. 

9. An apparatus for reducing noise in a video signal which 
includes a plurality of video frames of digitized video 
information represented in a plurality of pixels, the appara- 
tus comprising: 

means for storing a current video frame and a plurality of 

temporally adjacent video frames; 
means for selecting from the current video frame a current 

frame portion comprising a current pixel and at least 

one pixel spatially adjacent to the current pixel; 
means for selecting from the temporally adjacent video 

frames temporally adjacent frame portions comprising 

a pixel or pixels that spatially corresponds to the pixels 

of the current frame portion; 
means for determining the luminance value of each pixel 

in said current frame portion and said adjacent frame 

portions; 

means for determining the average luminance values of 
said current frame portion pixel and said spatially 
adjacent frame portions; 
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means for determining whether a local scene change 
occurs between said current frame portion and each of 
said adjacent frame portions by: 
means for determining and comparing to a first thresh- 
old value the absolute value of the difference 5 
between the average luminance of said current frame 
portion and the average luminance of said adjacent 
frame portion; 
means for, 

if the absolute value of the luminance difference exceeds 10 
said first threshold value, assuming there is a local 
scene change in the video information between said 
current frame portion and said adjacent frame portion; 
else 

determining and comparing to a second threshold value 15 
the absolute values of the difference between the values 
of a color component of said current pixel and the 
corresponding pixel of said adjacent frame portion; 
means for, 

20 

if the absolute value of the color component value dif- 
ference exceeds said second threshold value, assuming 
there is a local scene change in the video information 
between said current frame portion and said adjacent 
frame portion; else 25 

assuming that there is no local scene change between said 
current frame portion and said adjacent frame portion; 

means for selecting an even number exceeding or equal- 
ing zero of adjacent frame portions considered to 
involve no local scene change in relation to said current 30 
frame portion; 

means for selecting the video information from a pixel 
that is probable to have equal predecessor and/or suc- 
cessors by: 

comparing the luminance values of said current pixel 35 
and the corresponding pixels in said selected adja- 
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cent frame portions and selecting the video informa- 
tion of the pixel having the median value of said 
luminance values; 
means for assigning said selected video information to 
the current pixel to thereby produce a video frame 
wherein noise has been reduced. 

10. The apparatus according to claim 9, comprising means 
for replacing the video information of the current video 
frame by the selected video information. 

11. The apparatus according to claim 9, comprising means 
for producing a new current video frame from the selected 
video information. 

12. The apparatus according to claim 10, wherein said 
plurality of temporally adjacent video frames are a prede- 
termined number of video frames symmetrically preceding 
and/or succeeding a current video frame. 

13. An apparatus for use in an image signal coding system 
for coding an input image signal into a compressed output 
signal, the apparatus comprising: 

means for receiving an input image signal devised to 
perform the method steps comprising digitized image 
data; 

adapted to perform the steps in accordance with claim 1; 

means for coding the image signal into a resulting com- 
pressed image signal, e.g. according to an MPEG 
compression scheme. 

14. A computer program product for use in a computer 
system adapted for reducing noise in a video signal which 
includes a plurality of video frames, comprising: 

a recording medium; 

means, recorded on the recording medium, in accordance 
with the means of claim 1. 

***** 



03/04/2004, EAST Version: 1.4.1 



United States Patent m 

Oyama et al. 



ill 



US005099329A 

[11] Patent Number: 
[45] Date of Patent: 



5,099,329 
Mar . 24, 1992 



[54] NOISE REDUCTION METHOD AND 
DEVICE FOR IMAGE SIGNAL 

[75] Inventors: Koichi Oyama; Xiaohong Sun, both of. 
Tokyo, Japan 

[73] Assignee: Graphic Communication Tech., Ltd*, 
Tokyo, Japan 

[21] Appl.No.: 686,164 

[22] Filed: Apr, 16, 1991 

[30] Foreign Application Priority Data 

Apr. 17, 1990 [JP] Japan 2-101417 

Apr. 26, 1990 [JP] Japan 2-110959 

Jan. 9, 1991 [JP] Japan 3-12751 

[51] Int. a.s H04N 5/213 

[52] US. CL 358/167; 358/36 

[58] Field of Search 358/167, 166, 36, 37, 

358/168, 177, 155, 156, 157 

[56] References Cited 

U.S. PATENT DOCUMENTS 

4,249,210 2/1981 Storey .... 358/167 

4,539,594 9/1985 IUetschko : 358/167 

4,549413 10/1985 Metschko 358/167 

4,639,784 1/1987 Fling 358/167 

4,652,907 3/1987 Fling « 358/36 

4,689,671 8/1987 Ohki :. 358/167 

4,796,088 1/1989 Nishimura 358/167 

4,833,537 5/1989 Takeuchi 358/167 

5,025,316 6/1991 Darby : 358/167 

Primary Examiner— John W. Shepperd 
Assistant Examine r— Sherrie Hsia 



Attorney, Agent, or Firm — Iandiorio & Dingman 



[57] 



ABSTRACT 



The Erst difference signal between the current and 
previous frame signals is obtained by the subtracter 
pixel by pixel and supplied to both the line memory and 
the average value calculation circuit The average value 
calculation circuit calculates the average value for at 
least one line of the first difference signal which relates 
to only pixels with smaller absolute of the difference 
value than the present threshold. If the pixels which 
have absolute values smaller than the threshold, are less 
than the present number, the average value is set 0. 
Large values of the first difference caused by motion of 
the target are excluded and not treated as the flicker 
noise. The second difference signal is transformed by 
the non-linear circuit according to predetermined char- 
acteristics and supplied to the adder. The previous 
frame signal is supplied to the adder at the same time by 
delaying the signal through the line memory. The adder 
adds the output signals of the non-linear circuit and the 
line memory at each same pixel, and outputs to the 
output port and the frame memory. In a second embodi- 
ment the noise reduction device may include a noise 
reduction circuit and evaluation calculation circuit to 
decrease signal distortion for more natural image qual- 
ity. In a third embodiment the noise reduction device- 
may include a discrimination circuit and selectively 
switchable multipliers to adjust factors to optimize 
noise reduction or distortion reduction based on human 
visual sensitivity. 

5 Claims, 7 Drawing Sheets 
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changes continuously corresponding with the result of 

NOISE REDUCTION METHOD AND DEVICE FOR discrimination for each pixel. 

IMAGE SIGNAL As another example, the center pixel is pointed to be 

noise-reduced at first. All interframe absolute differ* 

BACKGROUND OF THE INVENTION 5 ences near the center pixel are compared with given 

1. Held of the Invention constants, and motive/static discrimination is made 
This invention is related to a digital noise reduction based, <> n these comparison. Then if discrimination des- 

device for digital image processing, and actually intends ignates static conditions, noise-reduction value will be 

to provide a noise-reduction filter as preprocessing with increased. However* If discrimination designates mo- 

interframe coding based digital video codes for better 10 tive conditions, noise-reduction value will be decreased, 

image fidelity and quality. (Japanese Patent Laid-Open Publication No. 2-7773). 

2. Prior Art Third Prior Art 

First Prior Art Next, FIG. 10 is a block diagram showing the con- 

FIG. 7 is a block diagram showing an example of the struction of an another Prior Art In FIG. 10, a digitized 

refilter wellknown. A subtracter 2 subtracts an previous 15 input image signal S6 is supplied to a subtracter 8 

image signal S5, which is stored in the frame memory 5, through an input port 7. At this subtracter 8, a frame 

from an original image input signal SI, which is input memory data S10 is supplied from the frame memory 

from the input port 1, pixel by pixel. As the result, a 11, So that, an interframe difference S7 is obtained at 

difference signal S2 is outputted from the subtracter 2, the subtracter 8 as difference between a previous image 

and supplied to a non-linear circuit 3. The difference 20 ^ original input signal An interframe difference 

signal S2 is transformed according to characteristics of signal S7 is supplied to a multiplier 9, and is multiplied 

the non-linear circuit 3. by multiplication coefficient K to obtain a multiplied 

Transformed difference value S3 is supplied to an ngnai S8. Then the multiplied signal S8 and the frame 

adder 4, wherein the difference value S3 is added with memory output signal S10 are supplied to the adder 10, 

the previous frame signal S5 which designates a prcvi- 25 ^ ^ added ^he adder 10 is outputted a output image 

ous frame image from the frame memory 5. The adder §9 to an output port 12 and the frame memory 11. 

4 outputs a result of addition as an additional signal S4 ( ^ ^ frame memory llf one fraine c f j^gg data is 

then this additional signal S4 is outputted from the out- ^orcd. Generally, the multiphcation coefficient K at 

put port 6, and is also stored into the frame memory 5. ^ multi Uer 9 ^ j or less ^ ^ & usually applied to 

In the above-menttoned operation, for example, the 30 ^ ^ mam& Qf ^ ffiotion detector 
non-linear cuxuit 3 has charactensncs which suppress desc ^ iQ £ fi e) which dctects a motioil of me 
the amplitude of an input signal by non-^eamtion coefficient K for the each pixel is different 
when the absolute value of the amphtude of the mput * multiplied in the multiplier 9. 
agr^issmaU* According to the noise reduction device described in 
8. In other words, the mput signal SI includes noise, 35 . "r*' .*"Jr, . ~7 • 4 , „u--~u**A-~f M 
such as Fhcker noise relited to fluorescent lights or *f **> ve ' * l™ G r 7 - * ~ 
camera noise, which has generally smaller amplitude mterframe difference signal S2 is smaUer than the preset 
than an interframe difference signal. So that, smaller value VI, this signal « is suppressed as noise. There- 
noise signals than the preset valie VI in terms of the f ore P reset valuc V J * " ™»H P^ible 
absolute amplitude are suppressed through the non-lin- .40 m order to reconstruct high fidelity of image, 
ear circuit 3 as shown in FIG. 8. Therefore, the preset . However, m the cases of videoWerences and 
value VI is usually preset at almost the maximum value . video-phones as m-house use, most of lights are fluores- 
of included noise in the input signal SI. cent ones, and cheap TV cameras are usually utilized, so 

Behavior of noise suppression by the non-linear cir- ^ tevd of noise which is caused by above-men- 

cuit 3 will be explained by using FIG. 9. In FIG. 9, the 45 tioned circumstances, is relatively large. Thus, the pre- 

solid line expresses the output signal S2 of the subtracter M value VI has to be set as relatively larger. Conse- 

2 in FIG. 7, which includes the flicker noise ingredient quently, if the difference signal S2 is a normal signal but 

denoted by the dashed line. And, reference A denotes «<>t noise, such as the signal of period Bl in FIG. 9, is 

the period of random noise, and B denotes the period of suppressed as noise. 

motion of objects, respectively. If the preset value VI in 50 As explained above, in this first prior art using the 
the non-linear circuit 3 is preset at the larger value than prefilter technique in FIG. 7, the signal of motion part 
the maximum value like the dashed dotted line, all sig- which has the small difference value of the motion 
nals whose absolute amplitude are smaller than VI are image signal, is not reconstructed with high fidelity, 
suppressed as noise, including effective signals during thus the image becomes unnatural, because the effective 
Bl period. 55 signals are sometimes suppressed. 
Second Prior Art According to two examples of the motive/static dis- 
Pre-filtering mettiods, that includes motive/static crimination techniques described before, these tech- 
discrimination about the image, have been proposed. niques can obtain better results than the motive/static 
An example of them is the method which decides the evaluation based on each pixel's interframe. difference 
noise reduction characteristics after discriinination of 60 only. Because the possibility of descrimination-mis- 
not only the interframe differences but also neighbor takes, in which larger amplitude signals in the static area 
pixels differences for, motive/static evaluation. (Japa- are discriminated as signals in the motive area, de- 
nese Patent Laid-Open Publication No. 1-143583). creases. 

In this methods, the interframe absolute differences However, even if these techniques are adopted, rela- 

are used for motive/static discrimination, which con- 65 ti very large noises sometimes make the motive/static 

. sists of the target pixel's interframe absolute difference discrimination mistaken for the supplied image signal, 

and the summation of the neighbor pixel's interframe In the first case, for example, when the motive/static 

absolute differences. Noise reduction characteristics evaluation value, X f i=>|X 0 |-f|Xi| + IX2I + IX3I+ . . 
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. + |Xn| is smaller than the preset threshold or the tively using area of the digital image processing to re- 
adaptive threshold changing according to | Xo | and X s \ t duce noise of video image signal, 
the target pixel is discriminated as a pixel in the static In an aspect of the present invention to provide a 
area, where X* is the target pixel's interframe difference, noise reduction device including: ' 
and Xi, X* . . . Xn are neighbor pixels' interframe dif- 5 a first subtracter means for obtaining a first difference 
ferences, respectively. between a current frame signal and a previous frame 

However, noise which includes higher frequency signal pixel by pixel; 

spectrum makes these difference values Xi, X 2 , X 3 , . . . a first line memory means for outputting said first 

, Xn, both positive and negative. Therefore, if absolute difference with a delay time corresponding to at least 

summation of them are used for motive/static evalua- 10 one line processing time; 

tion, the threshold must be set relatively larger. Conse- an average value calculation means for calculating an 

quently a motive signal with a small interframe differ- average value of said first difference which is in a preset 

ence could be discriminated as a signal in the static area, ^ on * ,east one a™ of P™* 

and be redacted as noise, because of too strong noise * second subtracter means for obtaining a second 

reduction effect 15 dlfferencc between an output of said first line memory 

As the result, even effective motive signals sometimes means ^ « of avera * e value calculation 

suffer from distortion, which brings unnatural artifact means P« cl bv P""* . . . . e . 

such as residual images. a ^ta^rmeam for suppressing said second differ- 

On the other hand in the second case, when allabso- ence smaller than a preset value; 

viiuicuuiw ^.1. v v v v« r,r^u 20 a second Ime memory means for outputtmg said pre- 

lute of mterframe vious frame signal with a delay time to output at same 

neighbor pixel, are smaUer than the prese : **b« ^ ^ ^ rf-d m& J 

that is, X<2=max(|Xi| .1*1. ■ - V^!> x " ™5 anldder means for adding an output of said non-lin- 

than die preset threshold, the center point of the target ^^^^ om t jf ^ ^ ^ m 

area is discnrmnated as the pixel in the static area. „ ^ ^ ^ £ d 

Therefore, it is necessary for this preset threshold to be a f ^ e mem ^ means for mem0 rizing an output of 

set relatively larger in order to suppress higher fre- ^ ^ mean$v ^ for outputting ^ previous 

quency noise. Consequently this second case has also frame signal> which ^ memo rized therein, to said first 

the same problem as first case. subtracter means and said second line memory means. 

In the case of the third prior art showing FIG. 10, 3Q ^ a resuU) acting to ^ f irst aspect of the inven- 
when the coefficient K is exactly the same as 1, the tfon> ^ average value of the original frame and the 
interframe difference S7 is directly sent to the output pre vious frame about one line or several lines are calcu- 
port 12 as an output image signal S9. If the input image kted pixel by pixe1 based on ^ fj uor escent light's 
signal S6 includes noise at this time, the noise is also flicker noi$c j ower frequency (100 Hz or 120 Hz), 
looked as a part of interframe difference S7 from the 35 because the intensity are almost DC ingredient and does 
stored data in the frame memory 11, and is supplied to not change along the same scanning line. The smaller 
the multiplier 9. At the multiplier 9, the noise is multi- interframe difference value after subtraction of the in- 
plied by the coefficient K(= 1), then it directly appears terframe difference by this average value as the flicker 
at the output port 12. In this case, noise reduction can- m use< j p and the suppression threshold is also set 
not be provided. 40 smaller corresponding to smaller by the non-linear cir- 

When the coefficient K is smaller than 1, a difference C uit. 

in an input image signal S6 appears as the value multi- As it is clarified by these explanation, the non-linear 

plied by K in the output image signal S9, so that this circuit can suppress random noise ingredients only, 

difference in the input image signal S6 affects the output because the amplitudes of the 2nd. Interframe differ- 

image signal S9 several frames later only to reconstruct 45 ence signals are gotten by subtracting the average value 

the image with some fidelity. 0 f the 1st. Interframe difference signals between the 

Therefore the smaller the coefficient K is, the more a original frame and the previous frame as the flicker 

random type of signal such as noise is suppressed as noise from the original frame signal. There fore possibil- 

noise reduction effect. However even effective differ- ity of suppression of effective signals decreases, and 
ence from the previous frame appears to be recon- 50 fidelity of image reconstruction is improved especially 

structed several frames later with some fidelity, which for relatively small motion signals, 

makes the distortion that trails a tail in the image, called in a second aspect of the present invention to provide 

"comet tail". Then the multiplier 9 is operated in order a noise reduction device including: 

that the coefficient K is set smaller than 1 for almost a subtracter means for obtaining a first difference 

static image to perform noise reduction effect, and is set 55 between a current frame signal and a previous 

1 or close to 1 for motive image. frame signal pixel by pixel; 

In other words, the coefficient K is set only by mo- a plural pixel memory for obtaining second differ- 

tion independent of the image brightness. ences of a target pixel for noise reduction and of 

As the result, when the difference from the previous neighboring pixels of said target pixel, based on 

frame is relatively large, noise reduction effect decrease 60 said first difference; 

because the coefficient K is set a value close to 1. On the an evaluation value calculation means for calculating 

other hand, the distortion called "comet tail" conspicu- absolute values of summation of said first differ- 

ously appears according to increase of noise reduction ence and said second differences, as evaluation 

effect. value for detennining a noise reduction character- 

65 istic; and 

SUMMARY OF THE INVENTION a noise reduction means for suppressing noise includ- 

Accordingly, it is a purpose of the present invention ing said first and second differences of pixels based 

to provide a noise reduction device capable of effec- on said noise reduction characteristic 
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In a third aspect of the present invention to provide a 

noise reduction device including: 
a subtracter means for obtaining a first difference 
between a current frame signal and a previous 
frame signal pixel by pixel; 5 
first pixel memory means for obtaining second differ* 
ences of a part of pixels which are a target pixel for 
noise reduction and neighboring pixels of said tar- 
get pixel, based on said first difference; 
noise reduction means for suppressing noise including 10 
said first differences based on an evaluation value 
for determining a noise reduction characteristic; 
second pixel memory means for obtaining noise re- 
duced differences, by said noise reduction means, 
of rest part of said pixels which are a target pixel 15 
for noise reduction and neighboring pixels of said 
target pixel based on said first difference; and 
an evaluation value calculation means for calculating 
absolute values of summation of said noise reduced 
differences from said second pixel memory means 20 
and said first and second differences of at least each 
neighboring pixel, as said evaluation value for de- 
termining said noise reduction characteristic. 
As a result, according to these second and third as- 
pects of the invention, the absolute value of the summa- 25 
tion of the interframe difference values of the target 
pixel and its neighbor pixels or the absolute value of the 
summation of the interframe difference values of neigh- 
bor pixels except the target pixel are used for motive/- 
static evaluation. Characteristics of noise reduction is 30 
decided based on this evaluation value. And also the 
interframe difference values after noise reduction are 
included as values for evaluation value calculation. 

By this means, it is possible for the motive/static 
thresholds about the target pixels for noise reduction to 35 
set smaller values, which makes the motive/static eval- 
uation more accurate even for relatively smaller inter- 
frame difference signals. 

As the threshold for motive/static discrimination 
about as target pixel can be set smaller, possibility of 40 
miss-discrimination based or too strong noise reduction 
effect, which looks a small interframe difference signal 
in the motive area as one in the static area, decreases. 
There fore distortion of effective signals decreases, and 
brings more natural image quality. 45 

In a forth aspect of the present invention provide a 
method for reducing noise of image signal, comprising 
steps of: 

obtaining a difference signal between a input image 
signal which is a current frame signal and a memo- 50 
rized signal which is a previous frame signal pixel 
by pixel; 

obtaining a multiplication coefficient which is smaller 
than 1, in case of that brightness of image desig- 
. hated by said current frame signal has been chang- 55 
ing toward the . brighter when said brightness is 
bright, and in case of that said brightness has been 
changing toward the darker when said brightness is 
dark, on the basis of said input image signal and 
said difference signal, 60 

however, obtaining a multiplication coefficient 
which is less than 1 and is larger than said multipli- 
cation coefficient, in case of that said brightness has 

. been changing toward the darker when said bright- 
ness is bright, and in case of that said brightness has 65 
been changing toward the brighter when said 
brightness is dark, on the basis of said input image 

. signal and said difference signal; 



obtaining a multiple signal by multiplying said differ- 
ence signal by said multiplication coeficient; 

obtaining summation of said memorized signal and 
said multiple signal; 

then storing said summation as a memorized signal 
for input image signal of next frame, and outputting 
said summation as final output image signal at the 
same time. 

In a fifth aspect of the present invention provide a 
noise reduction device including: 

a subtracter means for obtaining a difference between 
a current frame signal and a previous frame signal; 

discriminator means for discriminating that said dif- 
ference is multiplied a multiplication coefficient 
which is smaller than 1, in case of that brightness of 
image designated by said current frame signal has 
been changing toward the brighter when said 
brightness is bright, and in case of that said bright- 
ness has been changing toward the darker when 
said brightness is dark, on the basis of said input 
image signal and said difference signal, 

however, that said difference is multiplied a multipli- 
cation coefficient which is less than 1 and is larger 
than said multiplication coefficient which is smaller 
than 1, in case of that said brightness has been 
changing toward the darker when said brightness is 
bright, and in case of that said brightness has been 
changing toward the brighter when said brightness 
is dark, on the basis of said input image signal and 
said difference signal; 

multiplier means for obtaining a multiply signal by 
multiplying said difference signal by said multipli- 
cation coefficient according to said discriminating; 

adder means for obtaining an output image signal by 
adding said memorized signal and said multiply 
signal; and 

frame memory means for storing said output image 
signal, and for outputting said output image signal 
memorized therein as said memorized signal for 
next frame. 

As a result, according to these forth and fifth aspects 
of the invention is based on human visual, sensitivity. 
When human visual sensitivity is low, and high, the 
coefficient K is set smaller than 1 for higher noise re- 
duction effect and is set close to 1 for higher distortion 
reduction, respectively. 

Human eyes are not so sensitive for distortion called 
"comet tail" when the frame changes from a bright one 
to a brighter one. There is the same situation in the case 
that the frame changes from a dark one to a darker one. 
On the other hand, human eyes become more sensitive 
for distortion when the frame changes from a bright one 
to a dark one or opposite. There fore higher noise re- 
duction effect can be achieved with the minimum dis- 
tortion by this means. 

As the distortion called "comet tail*' can be excluded 
by changing the multiplier coefficient based on human 
visual sensitivity for brightness level, effective noise 
reduction becomes available with the minimum distor- 
tion. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram showing a noise reduction 
device of the first embodiment; 

FIG. 2 is a waveform chart to explain the operation 
about behavior of noise suppression by the nonlinear 
circuit based on FIG. 1; FIG. 3 is a block diagram 



03/04/2004, EAST version: 1.4.1 



5,099,329 

7 8 

showing a noise reduction device of the first example in 28 adds the output signal S17 of the non-linear circuit 27 

the second embodiment; and the output signal S16 of the line memory 26 at each 

FIG. 4 is a block diagram showing a noise reduction same pixel, 

device of the second example in the second embodi- The added value S18 from the adder 28 is outputted 

ment; 5 from the output port 30, and is supplied to the frame 

FIG. 5 is conception diagram showing positions of memory 29 and stored. The above-mentioned opera- 
each pixel to explain arrangement of pixels which are tions are repeated. 

elements of evaluation value by the evaluation value Next, noise suppression will be explained by using 

calculation circuit of FIG. 3; FIG. 2, where the signal described by the solid line SIS 

FIG. 6 is a block diagram showing a noise reduction 10 in FIG. 2, corresponds to the signal S2 in FIG. 9, and 

device of the third embodiment; the area A, B correspond the area A, B in FIG. 9, re- 

FIG. 7 is a block diagram showing a conventional spectively. 

noise reduction device of the first prior art; In FIG. 2, the amplitude of the second interframe 

Fig. 8 is an example of input-output characteristic of difference signal SIS, which is supplied to the non-lin- 

the non-linear circuit in FIG. 7; 15 ear circuit 27 (see FIG. 1), is subtracted the average 

FIG. 9 is a waveform to explain behavior of noise value signal S14 as the flicker noise included in the input 

suppression by the non-linear circuit in FIG. 7; signal Sll from the first interframe difference signal 

FIG. 10 is a block diagram showing a conventional S13. Therefore, the amplitude of the signal decreases by 

noise reduction device of the third prior art the amplitude P of the flicker noise compared with the 

tnco^iwt^m toc imrnrDocn 20 S2 in FIG. 9. As the result, the preset value V2 
DESCRIPTION OF TOE PREFERRED (douWe dotted line) for the ^4^^ circuit 27 can be 

EMBODIMENT set smaller than the former preset value VI (dashed 

Hereinafter, embodiments of the present invention line) corresponding to the maximum wave height value 

will be described by reference to drawings. of noise. Then, the effective signal only during the per- 

First embodiment 25 iod B2 in the period B cannot be distinguished from 

FIG. 1 is a block diagram showing a noise reduction noise to be suppressed. However, the situation is im- 

device of the first embodiment of this invention. proved for less distortion compared with the former 

In FIG. 1, the first interframe difference signal S12 case that the effective signal during the period Bl (see 

between the original frame signal Sll which is input FIG. 9) is treated as noise. 

from the input port 21 and the previous frame signal S19 30 Second embodiment 

which is output from the frame memory 29 is obtained FIG. 3 is a block diagram showing a noise reduction 

by the subtractor 22 pixel by pixel. The obtained first device of the second embodiment of this invention, 

interframe difference signal S12 is supplied to both the where the symbols of the components correspond to 

line memory 23 and the average value calculation cir- those in FIG. 1. And, FIG. 5 is a conception diagram 

cuit 24. The average value calculation circuit 247 calcu- 35 showing positions of each pixel to explain arrangement 

lates the average value for one line or several lines of of pixels which are elements of evaluation value by the 

the first interframe difference signal S12, where the evaluation value calculation circuit of FIG. 3. 

average values are calculated for only pixels with In FIG. 5, each circle denotes each pixel, where PO 

smaller absolute of the interframe difference value than is the target pixel to be suppressed noise, and the neigh- 

the preset threshold. And, in pixels of one line or several 40 bor pixels, such as Pi, P2, Pa and P4, are utilized noise 

lines, if the pixels which have absolute values smaller reduction processing. 

than the threshold, are less than the preset number, the In FIG. 3, the subtracter 22 generates an interframe 

average value is set 0. Because the average value calcu- difference signal pixel by pixel by subtracting a previous 

lated by the average calculation circuit 24 is looked as frame signal S29 which is stored in the frame memory 

the flicker noise in this invention, and large values of the 45 29, from an original frame signal Sll is input at the input 

first interframe difference S12 caused by motion of the port 21 at first. The difference signal S20 obtained by 

target are excluded not to be looked as the flicker noise. the subtractor 22 designates a difference of the pixel P4, 

After the average value of the first interframe differ- and is supplied to both the line memory 23 and the 
ence S12 whose absolute values of one or several lines evaluative value calculation circuit 39 for motive/static 
which are smaller than the threshold is calculated by 50 discrimination. The line memory 23 provides the input 
the average value calculation circuit 24 in this manner, difference signal S20 with the delay time corresponding 
the subtracter 25 subtracts the line memory output sig- to one pixel processing time shorter than one line. The 
nal S13 from the line memory 23 by the average value difference signal S21 delayed by the line memory 23 
to obtain the second interframe difference signal S15, designates a difference of the pixel P3, and is supplied to 
which is transformed by the non-linear circuit 27 ac- 55 both the pixel memory 35 and the evaluation value 
cording to predetermined characteristics. The second calculation circuit 39. The pixel memory 35 provides 
interframe difference signal S17 is supplied to the adder the input signal S21 with the delay time corresponding 
28. At the same time, the previous frame signal from the to one pixel processing time. The difference signal S22 
frame memory 29 is delayed through the line memory delayed by the pixel memory 35 designates a difference 
26, and the output signal S16 of the line memory 26 is 60 of the target pixel PO, and is supplied to the pixel mem- 
supplied to the adder 28. ory 37, the evaluation value calculation circuit 39 and 

As the delay time of the subtracter 25 and the non-lin- the noise reduction circuit 36. The pixel memory 37 

ear circuit 27 is negligible small, the delay time of the provides the input signal S22 with the delay time corre- 

line memory 26 correspoonds to that of the line memory sponding to one pixel processing time just like the pixel 

23. Hence the delay time of the frame memory 29 is 65 memory 35. The difference signal S30 delayed by the 

equivalent to the value which is subtracted the delay pixel memory 37 designates a difference of the pixel P2, 

time of the line memory 26 or 23 from that of the con- and is supplied to the line memory 38 and the evaluation 

ventional frame memory 5 (see FIG. 7). Then the adder value calculation circuit 39. The line memory 38 pro- 
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vides the imput signal S30 with the delay time corre- discrimination as evaluation value, the threshold for 

sponding to pixel processing time just like the line mem- . discrimination can be set much smaller than in the case 

ory 23. which has been already explained before. The of the first and second prior art, where the difference, 

difference signal S31 delayed by the line memory 38 value Xs\ (first prior art) or difference value X$i (sec- 

designates a difference of the pixel PI, and is supplied to 3 ond prior art) is. used as evaluation value, 

the evaluation value calculation circuit 39. When this threshold is set relatively small, it is possi- 

Thus, interframe difference signals S22, S31, S30, S21 ble for small difference of motion signals to avoid to be . 

and S20 about each pixel PO~P4 are generated by the discriminated as static signals with high noise reduction 

subtracter 22, and are supplied to the evaluation value effect As a result, the distortion of small effective mo- 
calculation circuit 39, where an evaluation value for to tion signal decreases, and a more natural and high qual- 

motive/static discrimination about the target pixel PO is j ty \ mB g t can be obtained. 

calculated by using the input signals S22, S31, S30, S21 ^ ^ other pjQ 4 shows a block diagram 0 f 

and S20. And, the absolute values which are summation ^ other cxamp i e 0 f th c second embodiment in this 

of each difference signal S22, S31, S30, S21 and S20 are mventioilf whcrc the same symbols are used for corre- 

used as evaluation values. . u s din components. 

In other words, as the absolute value of the summa- ^ ereinafter| only different part from the example in 

tion. Xs= l^+Xi+*+Xs+X4|. is calculated, p, G 3 ^ ^ lained by FIG. 4. The output 

where each X* Xi, X h X 3 and X4 is the difference ^ S24 rf ^ ^ memo 37 md ^ ou ^ ut rignal 

corresponding with 1 the pixel Pl-the differences^ S »of the line memory 38aresupphed to theevaluation 

S31, the pad P2-the difference signal S30, the pixel 20 ,„i... „i„.i„..„ _:„.,:, «n „ ft „ „,„ 

P3-flie difference signal S21. and the pkel P4-the d&er. » after noise ^toon pro- 

a w-mt uut«u»^« "o""^**' , , , i- cessins bv the noise reduction circuit 36. Then the eval- 

ence signal S20, respectively. After Xi is calculated, this ~T^ K y , . ^ " T . ' ' " " TT " 

result Compared with the preset threshold or the v f^f J? C ** U ™ J|J£ S2? 

adaptive threshold dependent of Xsand |Xo| for.moti- I*?** i+X ? +X 3 +X4ias evaluation value for moti- 

ve/static a^rimination. Then, the evaluated signal S26 25 ™'*™ c discrimination, where each ^ X3 and X* b a 

is supplied to the noise reduction circuit 36. difference value based on each signal S22. S21 and S20 

ThT noise reduction circuit 36 provides the noise respectively, and where each X 1 and X 2 is ,* 'difference 
reduction processing according to the evaluation result va3ue ° T n ^ n01sc re ? uced S25 and S24 
about the difference signals S22 which designates a respectively. In this case, as the mentioned above, gen- 
difference of the original target pixel PO. Thus the dif- 30 frequency noise consists of both positive 
ference signal S23 about the target pixel PO, which is negative values of each difference value X6» X'i. v 
redacted noise, is supplied to the adder 28. At adder 28, X'2, X3 and X4. 

the difference signal S23 is added the output signal S27 Then the similar expression to FIG. 3 is available as 

of the line memory 26, which signal is delayed signal of follows. 

the output signal S29 of the frame memory 29 origi- 35 

nally. The result signal S28 of the addition is outputted *V< *ii. 

from output port 30, and is also supplied to the frame 

memory 29 and stored. The delay time by the line mem- where - yT| _ 

ory 26 correponds to the one line processing. The delay w+^i+^a+w-^i-** 

time of the signal S22 which is outputted through the ^ 

line memory 23 and the pixel memory 35 also corre- Xg><Xa> 

sponds to the one line processing. Because the delay 

times by the evaluation value calculation circuit 39 and where 

the noise reduction circuit 36 are negligible short The |JTj+r 2+^3+^1 =JTjo. 

output signal S28 of the adder 28 is delayed correspond- 45 

ing to one line processing, so that the delay time of the Therefore, equivalent advantage to the former case in 
frame memory 29 is set one line shorter than one frame. pio 3 can be obtained by this approach, because 
Accordingly, in the second embodiment of this inven- smaller thresholds can be set than the cases of conven- 
tion, the absolute value of the summation of the differ- tional technologies. 

ence values Xo, Xi, X2, X3, Xiabbut the target pixel PO, 3Q ne ^ mcm bry 23, 38, the pixel memory 35, 37 and 

and the neighbor pixels P1~P4 are used as evaluation ^ noise rc< |uction circuit 36 are applicable for the 

value to obtain motive/static discrimination of the tar- former cases of conventional technologies as they are. 

get pixel PO. Generally, High frequency noise consists ^ thc described above, only the target pixel PO and 

of both positive and negative values Xo~X4 signals. its nd ghboring pixels P1-P4 have been used as ele- 

75 ere £? re - v v m v , . J* 1 ? Iv , w b ? lil ?£ n ^ 35 mcnts of evaluation calculating for the motive/static 

IX0+X1+X2+X3+X4I < |Xo| + |Xi| + |A2| + |X 3 | discrimination, however more neighbor pixels can be 

+ 1 X4 1 . In other formulation, ^ for evaluation value calculation corresponding to 

more line memories and pixel memories, of course. 

Jr '-' 1 Third Embodiment 

60 FIG. 6 is a block diagram showing a noise reduction 

|jr,+jr 2 +Xj+*4| <m« (|X,|.|jr2|.jr 3 |,|jK4|)x4. . device **rd embodiment in this invention, where 

the symbols of the components correspond to those in 

therefore, FIG. 10. 

X*<X& The signal Sll inputted from the mput port 21 is 

63 supplied to the subtracter 22 and the discrimination 44. 

where Xrf= IX1+X2+X3+X4I. The subtracter 22 subtracts the output signal S34 of the 

Thus, if the absolute of the summation of each differ- frame memory 29 from the input signal Sll. This differ- . 

ence value Xo~~X4 or Xi is used for motive/static ence signal S30 as a result is supplied to the multipliers 



or 
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41, 42 to be multiplied by the coefficients Ki and K2. 
The coefficient Ki is not bigger than 1, but is approxi- 
mate to 1. And, the coefficient K2 is positive and 
smaller than the coefficient KI. 

The output of either the multiplier 41 or the multi- 5 
plier 42 is selected by the switch 43 according to the 
discrimination signal S32, which is supplied from the 
discrimination 44, and is supplied to the adder 28 as the 
multiplier output signal S31. The adder 28 generates the 
output signal S33 by adding the mulitplier output signal 10 
S31 and the output signal S34 of the frame memory 29, 
and outputs it to both the frame memory 29 and the 
output port 30. 

Here, when an input signal Sll comes into the input 
port 21, the discriminator 44 discriminates brightness 15 
level of the input signal Sll whether it is bright or dark. 
At the next stage, the discriminator 44 discriminates 
polarity of a difference signal S30. In this case, there are 
four cases as follows. 

i) Input signal level: bright (changes from a bright con- 
dition to a brighter one) 

Difference signal polarity: positive 

ii) Input signal level: bright (changes from a bright 
condition to a darker one) ^ 
Difference signal polarity: negative 

iii) Input signal level: dark (changes from a dark condi- 
tion to a bright one) 

Difference signal polarity: positive 

iv) Input signal level: dark (changes from a dark condi- 3Q 
tion to a darker one) 

Difference signal polarity: negative 

Human vision sensitivity about the distortion called 
"comet tail 1 ' is not a sensitive for case i) and iv), and 
highly sensitive for case ii) and iii). Therefore, the dis- 35 
eliminator 44 outputs the discrimination signal S32 to 
select the coefficient KI which is close to 1, in the case 
of ii) and iii). However, in the case of i) and iv), the 
discriminator 44 outputs the discrimination signal S32 
to select the coefficient K2 which is smaller value than 40 
KI. By this means, noise reduction effect can be 
achieved with the minimum comet tail distortion. 

In the above-mentioned example, two multiplication 
coefficients Ki, K2 have been used, however it is possi- 
ble to increase the coefficients by dividing brightness 45 
into more levels, and considering how much difference 
signals change in addition to polarity, of course. The 
variable coefficient multiplier, where K is set by the 
discriminator 44, is also realized by using the multiplier 
9 in FIO. 10 in order to increase coefficients. Then the 50 
switch 43 can be omitted. 

What is claimed is: 

1. A noise reduction device including: 

a first subtracter means for obtaining a first difference 
between an current frame signal and a previous 53 
frame signal pixel by pixel; 

a first line memory means for outputting said first 
difference with a delay time corresponding to at 
least one line processing time; 

an average value calculation means for calculating an 60 
average value of said first difference which is in a 
preset range, on at least one line of pixels; 

a second subtracter means for obtaining a second 
difference between an output of said first line mem- 
ory means and an output of said average value 65 
calculation means pixel by pixel; 

a non-linear means for suppressing said second differ- 
ence smaller than a preset value; 



a second line memory means for outputting said pre- 
vious frame signal with a delay time to output at 
same timing with output of said non-linear means; 

an adder means for adding an output of said non-lin- 
ear means and an output of said second line mem- 
ory means pixel by pixel; and 

a frame memory means for memorizing an output of 
said adder means, and for outputting said previous 
frame signal, which is memorized therein, to said 
first subtracter means and said second line memory 
means. 

2. A noise reduction device including: 

a subtracter means for obtaining a first difference 
between an current frame signal and a previous 
frame signal pixel by pixel; 

a plural pixel frame memory for obtaining second 
differences of a target pixel for noise reduction and 
of neighboring pixels of said target pixel, based on 
said first difference; 

an evaluation value calculation means for calculating 
absolute values of summation of said first differ- 
ence and said second differences, as evaluation 
value for determining a noise reduction character- 
istic; and 

a noise reduction means for suppressing noise includ- 
ing said first and second differences of pixels based 
on said noise reduction characteristic. 

3. A noise reduction device including: 

a subtracter means for obtaining a first difference 
between an current frame signal and a previous 
frame signal pixel by pixel; 

fust pixel memory means for obtaining second differ- 
ences of a part of pixels which are a target pixel for 
noise reduction and neighboring pixels of said tar- 
get pixel, based on said first difference; 

noise reduction means for suppressing noise including 
said first differences based on an evaluation value 
for determining a noise reduction characteristic; 

second pixel memory means for obtaining reduced 
differences, by said noise reduction means, of rest 
part of said pixels which are a target pixel for noise 
reduction and neighboring pixels of said target 
pixel based on said first difference; 

an evaluation value calculation means for calculating 
absolute values of summation of said noise reduced 
differences from said second pixel memory means 
and said first and second differences of at least each 
neighboring pixel, as said evaluation value for de- 
termining said noise reduction characteristic. 

4. A method for reducing noise of image signal, com- 
prising steps of: 

obtaining a difference signal between a input image 
signal which is a current frame signal and a memo- 
rized signal which is a previous frame signal pixel 
by pixel; 

obtaining a multiplication coefficient which is smaller 
than 1 , in case of the brightness of image designated 
by said current frame signal has been changing 
toward the brighter when said brightness is bright, 
and in case of that said brightness has been chang- 
ing toward the darker when said brightness is dark, 
on the basis of said input image signal and said 
difference signal, 

however, obtaining a multiplication coefficient 
which is less than 1 and is larger than said multipli- 
cation coefficient, in case of that said brightness has 
been changing toward the darker when said bright- 
ness is bright, and in case of that said brightness has 



03/04/2004, EAST version: 1.4.1 



5,099, 

13 

been changing toward the brighter when said 
brightness is dark, on the basis of said input image 

. signal and said difference signal; 

obtaining a mulitple signal by multiplying said differ- . 
ence signal by said multiplication coefficient; 5 

obtaining summation of said memorized signal, and 
said mulitple signal; 

then storing said summation as a memorized signal 
for input image signal of next frame, and outputting 
said summation as final output image, signal at the 10 
same time. 

5. A noise reduction device including: 

a subtracter means for obtaining a difference signal 
imput image signal which between an is a current 
frame signal and a memorized signal which is a IS 
previous frame signal; 

discriminator means for discriminating that said dif- 
ference is multiplied a multiplication coefficient 
which is smaller than 1, in case of that brightness of 
image designated by said current frame signal has 20 
been changing toward the brighter when said 
brightness is bright, and in case of that said bright- 
ness has been changing forward the darker when 
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sid brightness is dark, on the basis of said input 
image signal and said difference signal, 

however, that 'said difference is roulitplied a mulit- 
plication coefficient which is less than 1 and is 
larger than said multiplication coefficient which is 
smaller than 1, in case of that said brightness has 
been changing toward the darker when siad bright- 
ness is bright, and in case of that said brightness has 
been changing toward the brighter when said 
brightness is dark, on the basis of said input image 
signal and said difference signal; 

mulitplier means for obtaining a multiply signal by 
multiplying said difference signal by said multipli- 

. cation coefficient according to said discriminating; 

adder means for obtaining an output image signal by 
adding said memorized signal and said multiply 
signal; and 

frame memory means for storing said output image 
signal, and for outputting said output image signal 
memorized therein as said memorized signal for 
next frame. 

* * * . * • 
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